Philian Mateo

Philian Mateo

1594009530

How to Create a Vanilla Neural Network with Tensorflow

Nowadays, Tensorflow is a highly demanded skill in the market, ensuring ease of production, standardizing some crucial stages of Machine Learning.

Today you’ll learn how to make your first neural network with Tensorflow; We’re going to build a Multilayer Perceptron model, also called the “Vanilla” Neural Network. Are you ready? So let’s start!

Tensorflow is a platform that focuses on Machine Learning, providing tools to create models for different ecosystems; With this, you can run Machine Learning models on mobile devices, cloud, servers, and more. Let’s check out the definition of Tensorflow by Google:

“TensorFlow is an end-to-end open source platform for machine learning. It has a comprehensive, flexible ecosystem of tools, libraries and community resources that lets researchers push the state-of-the-art in ML and developers easily build and deploy ML powered applications.”

It’s important to know that Tensorflow provides libraries and extensions for several situations; One of the most used is the Tensorflow Hub, which allows Machine Learning Engineers to reuse already trained models to train new ones, using the Transfer Learning method.

What is a Neural Network?

Generally, the easiest way to explain them is to say that they’re related to neurons in the animal brain but created by engineers, using the name Artificial Neural Network (ANN).

Designed to recognize patterns, they act like neurons, triggering when one is detected. Artificial Neural Networks are based on the animal’s Central Nervous System but not identical, just similar. It’s a common mistake when talking about neural networks.

Neural networks are composed of input, hidden and output layers; The Input Layer is the first, where our data is added; The hidden layers are the “neurons” that adjust the weight and bias for each epoch; Finally, it’s in the output layer that we receive a response from our model, based on the target.

Epoch is the number of times our Machine Learning model pass through the entire training data; Inside each epoch, the weight and bias are computed in order to make our model find better patterns.

If you’re new to Neural Networks, check out this Google playground; You can modify the number of epochs, activation, learning rate, and more (without breaking your model).

The Multilayer Perceptron

Today we will be building a Multilayer Perceptron (MLP), also called the “Vanilla” Neural Network; **By default, this type of MLP has only a single hidden layer **and looks like the gif above.

Being one of the simplest types of neural networks, the Multilayer Perceptron is perfect to understand how a neural network works and each step of the entire process.

The Google Colaboratory

We’ll use Google Colab, a tool based on the Jupyter Notebook that allows data scientists to work quickly and intelligently. By default, Google Colab comes with the necessary packages to work with Data Science and Machine Learning.

Also, it is a great tool for anyone who focuses on performance: Google offers a GPU that speeds up processes. Simply put,** you don’t have to worry about installing packages, machine memory, or backup**, Google saves your progress to your Google Drive account.

Structure of Today’s Project

To make it easier, we’ll work on a regression data; I’ll just focus on creating our Multilayer Perceptron, but you can click here to check out other steps, like Data Cleaning and Feature Engineering, for example.

About our Data

We’ll use the “Auto MPG” data provided by UCI, click here to take a look. We’ll predict the miles per gallon (MPG) parameter based on 7 different features.

#neural-networks #keras #machine-learning #data-science #tensorflow

What is GEEK

Buddha Community

How to Create a Vanilla Neural Network with Tensorflow
Mckenzie  Osiki

Mckenzie Osiki

1623135499

No Code introduction to Neural Networks

The simple architecture explained

Neural networks have been around for a long time, being developed in the 1960s as a way to simulate neural activity for the development of artificial intelligence systems. However, since then they have developed into a useful analytical tool often used in replace of, or in conjunction with, standard statistical models such as regression or classification as they can be used to predict or more a specific output. The main difference, and advantage, in this regard is that neural networks make no initial assumptions as to the form of the relationship or distribution that underlies the data, meaning they can be more flexible and capture non-standard and non-linear relationships between input and output variables, making them incredibly valuable in todays data rich environment.

In this sense, their use has took over the past decade or so, with the fall in costs and increase in ability of general computing power, the rise of large datasets allowing these models to be trained, and the development of frameworks such as TensforFlow and Keras that have allowed people with sufficient hardware (in some cases this is no longer even an requirement through cloud computing), the correct data and an understanding of a given coding language to implement them. This article therefore seeks to be provide a no code introduction to their architecture and how they work so that their implementation and benefits can be better understood.

Firstly, the way these models work is that there is an input layer, one or more hidden layers and an output layer, each of which are connected by layers of synaptic weights¹. The input layer (X) is used to take in scaled values of the input, usually within a standardised range of 0–1. The hidden layers (Z) are then used to define the relationship between the input and output using weights and activation functions. The output layer (Y) then transforms the results from the hidden layers into the predicted values, often also scaled to be within 0–1. The synaptic weights (W) connecting these layers are used in model training to determine the weights assigned to each input and prediction in order to get the best model fit. Visually, this is represented as:

#machine-learning #python #neural-networks #tensorflow #neural-network-algorithm #no code introduction to neural networks

Philian Mateo

Philian Mateo

1594009530

How to Create a Vanilla Neural Network with Tensorflow

Nowadays, Tensorflow is a highly demanded skill in the market, ensuring ease of production, standardizing some crucial stages of Machine Learning.

Today you’ll learn how to make your first neural network with Tensorflow; We’re going to build a Multilayer Perceptron model, also called the “Vanilla” Neural Network. Are you ready? So let’s start!

Tensorflow is a platform that focuses on Machine Learning, providing tools to create models for different ecosystems; With this, you can run Machine Learning models on mobile devices, cloud, servers, and more. Let’s check out the definition of Tensorflow by Google:

“TensorFlow is an end-to-end open source platform for machine learning. It has a comprehensive, flexible ecosystem of tools, libraries and community resources that lets researchers push the state-of-the-art in ML and developers easily build and deploy ML powered applications.”

It’s important to know that Tensorflow provides libraries and extensions for several situations; One of the most used is the Tensorflow Hub, which allows Machine Learning Engineers to reuse already trained models to train new ones, using the Transfer Learning method.

What is a Neural Network?

Generally, the easiest way to explain them is to say that they’re related to neurons in the animal brain but created by engineers, using the name Artificial Neural Network (ANN).

Designed to recognize patterns, they act like neurons, triggering when one is detected. Artificial Neural Networks are based on the animal’s Central Nervous System but not identical, just similar. It’s a common mistake when talking about neural networks.

Neural networks are composed of input, hidden and output layers; The Input Layer is the first, where our data is added; The hidden layers are the “neurons” that adjust the weight and bias for each epoch; Finally, it’s in the output layer that we receive a response from our model, based on the target.

Epoch is the number of times our Machine Learning model pass through the entire training data; Inside each epoch, the weight and bias are computed in order to make our model find better patterns.

If you’re new to Neural Networks, check out this Google playground; You can modify the number of epochs, activation, learning rate, and more (without breaking your model).

The Multilayer Perceptron

Today we will be building a Multilayer Perceptron (MLP), also called the “Vanilla” Neural Network; **By default, this type of MLP has only a single hidden layer **and looks like the gif above.

Being one of the simplest types of neural networks, the Multilayer Perceptron is perfect to understand how a neural network works and each step of the entire process.

The Google Colaboratory

We’ll use Google Colab, a tool based on the Jupyter Notebook that allows data scientists to work quickly and intelligently. By default, Google Colab comes with the necessary packages to work with Data Science and Machine Learning.

Also, it is a great tool for anyone who focuses on performance: Google offers a GPU that speeds up processes. Simply put,** you don’t have to worry about installing packages, machine memory, or backup**, Google saves your progress to your Google Drive account.

Structure of Today’s Project

To make it easier, we’ll work on a regression data; I’ll just focus on creating our Multilayer Perceptron, but you can click here to check out other steps, like Data Cleaning and Feature Engineering, for example.

About our Data

We’ll use the “Auto MPG” data provided by UCI, click here to take a look. We’ll predict the miles per gallon (MPG) parameter based on 7 different features.

#neural-networks #keras #machine-learning #data-science #tensorflow

Neural Networks: Importance of Optimizer Selection

When constructing a neural network, there are several optimizers available in the Keras API in order to do so.

An optimizer is used to minimise the loss of a network by appropriately modifying the weights and learning rate.

For regression-based problems (where the response variable is in numerical format), the most frequently encountered optimizer is the **Adam **optimizer, which uses a stochastic gradient descent method that estimates first-order and second-order moments.

The available optimizers in the Keras API are as follows:

  • SGD
  • RMSprop
  • Adam
  • Adadelta
  • Adagrad
  • Adamax
  • Nadam
  • Ftrl

The purpose of choosing the most suitable optimiser is not necessarily to achieve the highest accuracy per se — but rather to minimise the training required by the neural network to achieve a given level of accuracy. After all, it is much more efficient if a neural network can be trained to achieve a certain level of accuracy after 10 epochs than after 50, for instance.

#machine-learning #neural-network-algorithm #data-science #keras #tensorflow #neural networks

Marlon  Boyle

Marlon Boyle

1594312560

Autonomous Driving Network (ADN) On Its Way

Talking about inspiration in the networking industry, nothing more than Autonomous Driving Network (ADN). You may hear about this and wondering what this is about, and does it have anything to do with autonomous driving vehicles? Your guess is right; the ADN concept is derived from or inspired by the rapid development of the autonomous driving car in recent years.

Image for post

Driverless Car of the Future, the advertisement for “America’s Electric Light and Power Companies,” Saturday Evening Post, the 1950s.

The vision of autonomous driving has been around for more than 70 years. But engineers continuously make attempts to achieve the idea without too much success. The concept stayed as a fiction for a long time. In 2004, the US Defense Advanced Research Projects Administration (DARPA) organized the Grand Challenge for autonomous vehicles for teams to compete for the grand prize of $1 million. I remembered watching TV and saw those competing vehicles, behaved like driven by drunk man, had a really tough time to drive by itself. I thought that autonomous driving vision would still have a long way to go. To my surprise, the next year, 2005, Stanford University’s vehicles autonomously drove 131 miles in California’s Mojave desert without a scratch and took the $1 million Grand Challenge prize. How was that possible? Later I learned that the secret ingredient to make this possible was using the latest ML (Machine Learning) enabled AI (Artificial Intelligent ) technology.

Since then, AI technologies advanced rapidly and been implemented in all verticals. Around the 2016 time frame, the concept of Autonomous Driving Network started to emerge by combining AI and network to achieve network operational autonomy. The automation concept is nothing new in the networking industry; network operations are continually being automated here and there. But this time, ADN is beyond automating mundane tasks; it reaches a whole new level. With the help of AI technologies and other critical ingredients advancement like SDN (Software Defined Network), autonomous networking has a great chance from a vision to future reality.

In this article, we will examine some critical components of the ADN, current landscape, and factors that are important for ADN to be a success.

The Vision

At the current stage, there are different terminologies to describe ADN vision by various organizations.
Image for post

Even though slightly different terminologies, the industry is moving towards some common terms and consensus called autonomous networks, e.g. TMF, ETSI, ITU-T, GSMA. The core vision includes business and network aspects. The autonomous network delivers the “hyper-loop” from business requirements all the way to network and device layers.

On the network layer, it contains the below critical aspects:

  • Intent-Driven: Understand the operator’s business intent and automatically translate it into necessary network operations. The operation can be a one-time operation like disconnect a connection service or continuous operations like maintaining a specified SLA (Service Level Agreement) at the all-time.
  • **Self-Discover: **Automatically discover hardware/software changes in the network and populate the changes to the necessary subsystems to maintain always-sync state.
  • **Self-Config/Self-Organize: **Whenever network changes happen, automatically configure corresponding hardware/software parameters such that the network is at the pre-defined target states.
  • **Self-Monitor: **Constantly monitor networks/services operation states and health conditions automatically.
  • Auto-Detect: Detect network faults, abnormalities, and intrusions automatically.
  • **Self-Diagnose: **Automatically conduct an inference process to figure out the root causes of issues.
  • **Self-Healing: **Automatically take necessary actions to address issues and bring the networks/services back to the desired state.
  • **Self-Report: **Automatically communicate with its environment and exchange necessary information.
  • Automated common operational scenarios: Automatically perform operations like network planning, customer and service onboarding, network change management.

On top of those, these capabilities need to be across multiple services, multiple domains, and the entire lifecycle(TMF, 2019).

No doubt, this is the most ambitious goal that the networking industry has ever aimed at. It has been described as the “end-state” and“ultimate goal” of networking evolution. This is not just a vision on PPT, the networking industry already on the move toward the goal.

David Wang, Huawei’s Executive Director of the Board and President of Products & Solutions, said in his 2018 Ultra-Broadband Forum(UBBF) keynote speech. (David W. 2018):

“In a fully connected and intelligent era, autonomous driving is becoming a reality. Industries like automotive, aerospace, and manufacturing are modernizing and renewing themselves by introducing autonomous technologies. However, the telecom sector is facing a major structural problem: Networks are growing year by year, but OPEX is growing faster than revenue. What’s more, it takes 100 times more effort for telecom operators to maintain their networks than OTT players. Therefore, it’s imperative that telecom operators build autonomous driving networks.”

Juniper CEO Rami Rahim said in his keynote at the company’s virtual AI event: (CRN, 2020)

“The goal now is a self-driving network. The call to action is to embrace the change. We can all benefit from putting more time into higher-layer activities, like keeping distributors out of the business. The future, I truly believe, is about getting the network out of the way. It is time for the infrastructure to take a back seat to the self-driving network.”

Is This Vision Achievable?

If you asked me this question 15 years ago, my answer would be “no chance” as I could not imagine an autonomous driving vehicle was possible then. But now, the vision is not far-fetch anymore not only because of ML/AI technology rapid advancement but other key building blocks are made significant progress, just name a few key building blocks:

  • software-defined networking (SDN) control
  • industry-standard models and open APIs
  • Real-time analytics/telemetry
  • big data processing
  • cross-domain orchestration
  • programmable infrastructure
  • cloud-native virtualized network functions (VNF)
  • DevOps agile development process
  • everything-as-service design paradigm
  • intelligent process automation
  • edge computing
  • cloud infrastructure
  • programing paradigm suitable for building an autonomous system . i.e., teleo-reactive programs, which is a set of reactive rules that continuously sense the environment and trigger actions whose continuous execution eventually leads the system to satisfy a goal. (Nils Nilsson, 1996)
  • open-source solutions

#network-automation #autonomous-network #ai-in-network #self-driving-network #neural-networks

Sofia  Maggio

Sofia Maggio

1626106680

Neural networks forward propagation deep dive 102

Forward propagation is an important part of neural networks. Its not as hard as it sounds ;-)

This is part 2 in my series on neural networks. You are welcome to start at part 1 or skip to part 5 if you just want the code.

So, to perform gradient descent or cost optimisation, we need to write a cost function which performs:

  1. Forward propagation
  2. Backward propagation
  3. Calculate cost & gradient

In this article, we are dealing with (1) forward propagation.

In figure 1, we can see our network diagram with much of the details removed. We will focus on one unit in level 2 and one unit in level 3. This understanding can then be copied to all units. (ps. one unit is one of the circles below)

Our goal in forward prop is to calculate A1, Z2, A2, Z3 & A3

Just so we can visualise the X features, see figure 2 and for some more info on the data, see part 1.

Initial weights (thetas)

As it turns out, this is quite an important topic for gradient descent. If you have not dealt with gradient descent, then check this article first. We can see above that we need 2 sets of weights. (signified by ø). We often still calls these weights theta and they mean the same thing.

We need one set of thetas for level 2 and a 2nd set for level 3. Each theta is a matrix and is size(L) * size(L-1). Thus for above:

  • Theta1 = 6x4 matrix

  • Theta2 = 7x7 matrix

We have to now guess at which initial thetas should be our starting point. Here, epsilon comes to the rescue and below is the matlab code to easily generate some random small numbers for our initial weights.

function weights = initializeWeights(inSize, outSize)
  epsilon = 0.12;
  weights = rand(outSize, 1 + inSize) * 2 * epsilon - epsilon;
end

After running above function with our sizes for each theta as mentioned above, we will get some good small random initial values as in figure 3

. For figure 1 above, the weights we mention would refer to rows 1 in below matrix’s.

Now, that we have our initial weights, we can go ahead and run gradient descent. However, this needs a cost function to help calculate the cost and gradients as it goes along. Before we can calculate the costs, we need to perform forward propagation to calculate our A1, Z2, A2, Z3 and A3 as per figure 1.

#machine-learning #machine-intelligence #neural-network-algorithm #neural-networks #networks