The Machine Learning Crash Course โ€“ Part 2: Linear Regression

The Machine Learning Crash Course โ€“ Part 2: Linear Regression

The Machine Learning Crash Course โ€“ Part 2: Linear Regression - Welcome back to the second part of the Machine Learning Crash Course...๐ŸŒŸ๐ŸŒŸ๐ŸŒŸ๐ŸŒŸ๐ŸŒŸ

Welcome back to the second part of the Machine Learning Crash Course...๐ŸŒŸ๐ŸŒŸ๐ŸŒŸ๐ŸŒŸ๐ŸŒŸ

In the first part weโ€™ve covered the basic terminologies of Machine Learning and have taken a first look at Colab โ€“ a Python-based development environment which is great for solving Machine Learning exercises with Python and TensorFlow.

In this second part weโ€™ll move on and start with the first practical machine learning scenario which is solving a simple linear regression problem. First, letโ€™s clarify what linear regression is in general.

Linear Regression

The first Machine Learning exercise weโ€™re going to solve is a simple linear regression task. Linear regression is a linear approach to modelling the relationship between a dependent variable and one or more independent variables. If only one independent variable is used weโ€™re talking about a simple linear regression. A simple linear regression is what weโ€™ll be using for the Machine Learning exercise in this tutorial:

y = 2x + 30

In this example x is the independant variable and y is the dependant variable. For every input value of x the corresponding output value y can be determined.

Create A New Colab Notebook And ImporT Dependencies

To get started letโ€™s create a new Python 3 Colab notebook first. Go to https://colab.research.google.com login with your Google account and create a new notebook which is initially empty.

As the first step we need to make sure to import needed libraries. Weโ€™ll use TensorFlow, NumPy and Matplotlib. Insert the following lines of code in code cell:

from __future__ import absolute_import, division, print_function, unicode_literals
import tensorflow as tf
import numpy as np
import matplotlib.pyplot as plt

The first line of imports is just added because of compatibility reasons and can be ignored. The import statements for TensorFlow, NumPy and Matplotlib are working out of the box because all three libraries are preinstalled in the Colab environment.

Preparing The Training Data

Having imported the needed libraries the next step is to prepare the training data which should be used to train our model. Insert another code cell and insert the following Python code:

values_x = np.array([-10, 0, 2, 6, 12, 15], dtype=float)
values_y = np.array([10, 30, 34, 42, 54, 60], dtype=float)

for i,x in enumerate(values_x):
  print("X: {} Y: {}".format(x, values_y[i]))

Two NumPy arrays are initialised here. The first array (values_x) is containing the x values of our linear regression. This is the independent variable of y = 2x + 30. For each of the x values in the first array the second array (values_y) contains the corresponding y value.

By using a for-loop the value pairs are printed out:

If you like you can also use Matplotlib to visualise the the linear regression function as a graph:

x = np.linspace(-10,10,100) 
plt.title('Graph of y=2x+30') 
plt.plot(x, x*2+30);

Creating The Model

Next, weโ€™re ready to create the model (neural network) which we need to solve our linear regression task. Insert the following code into the notebook:

model = tf.keras.Sequential([
    tf.keras.layers.Dense(units=1, input_shape=[1])
])

model.compile(loss='mean_squared_error', 
              optimizer=tf.keras.optimizers.Adam(0.1))

Here weโ€™re using the TensorFlow-integrated Keras API to create out neural network. In order to create a new sequential model the tf.keras.Sequential method is used.

Note:

Keras is a high-level interface for neural networks that runs on top of different back-ends. Its API is user-friendly, yet flexible enough to build all kinds of applications. Keras quickly gained traction after its introduction and in 2017, the Keras API was integrated into core Tensorflow as tf.keras

In Keras, you assemble layers to build models. A model is (usually) a graph of layers. The most common type of model is a stack of layers: the* tf.keras.Sequential* model.

The call of the Sequential method is expecting to get an array (stack) of layers. In our case it is just one layer of type Dense. A Dense layer can be seen as a linear operation in which every input is connected to every output by a weight and a bias. The number of inputs is specified by the first parameter units. The number of neurons in the layer is determined by the value of the parameter input_shape.

In our case we only need one input element because for the linear regression problem weโ€™re trying to solve by the neural network weโ€™ve only defined one dependant variable (x). Furthermore the Dense layer is setup in the most simple way: it consists of just one neuron:

With that simple neural network defined itโ€™s easy to take a look at some of the insights to further understand how the neurons work. Each neuron has a specific weight which is adapted when the training is performed. The weight of every neuron in the fully connected Dense layer is multiplied with each input variable. As we only have defined one input variable (x) this input is multiplied with the weight w1 of the first and only neuron of the defined Dense layer. Furthermore, for each Dense layer a bias (b1) is added to the formula:

Now we can see why it is sufficient to only ad a Dense layer with just one neuron to solve our simple linear regression problem. By training the model the weight of the neuron will approach a value of 2 and the bias will approach a value of 30. The trained neuron will then be able to provide the output for inputs of x.

Having added the Dense layer to the sequential model we finally need to compile the model in order to make it usable for training and prediction in the next step.

The compilation of the model is done by executing the method model.compile:

model.compile(loss='mean_squared_error', 
              optimizer=tf.keras.optimizers.Adam(0.1))

Here we need to specify which loss function and which type of optimizer to use.

Loss function:

Loss functions are a central concept in machine learning. By using loss functions the machine learning algorithm is able to measure how much a prediction deviates from the actual result. Based on that determination the machine algorithm knows if the prediction results are getting better or worse.

The mean squred error is a specific loss function which is suitable to train a model for a linear regession problem.

As the name suggests, Mean square error is measured as the average of squared difference between predictions and actual observations. Due to squaring, predictions which are far away from actual values are penalized heavily in comparison to less deviated predictions.

Optimizer:

Based on the outcome which is calculated by the loss function the optimizer is used to determine the learning rate which is applied for the parameters in the model (weights and biases).

In our example weโ€™re making use of the Adam optimizer which is great for linear regression tasks.

Training The Model

The model is ready and the next thing we need to do is to train the model with the test data. This is being done by using the model.fit method:

history = model.fit(values_x, values_y, epochs=500, verbose=False)

As the first and the second argument weโ€™re passing in the test values which are available in arrays values_x and values_y. The third argument is the number of epochs which will be used for training. An epoch is an iteration over the entire x and y data provided. In our example weโ€™re using 500 iterations over the test data set to train the model.

After executing the training of the model letโ€™s take a look inside the development of the loss over all 500 epochs. This can be printed out as a diagram by using the following three lines of code:

plt.xlabel("Epoch Number") 
plt.ylabel("Loss Magnidute") 
plt.plot(history.history['loss'])

The result should be a diagram that looks like the following:

Here you can see that the loss gets better and better from epoch to epoch. Over the 500 epochs used for training weโ€™re able to see that the loss magnitude is approaching zero which shows that the model is able to predict values with a high accuracy.

Predicting Values

Now that the model is fully trained letโ€™s try to perform a prediction by calling function model.predict.

print(model.predict([20.0]))

The argument which is passed into the predict method is an array containing the *x *value for which the corresponding *y *value should be determined. The expected result should be somewhere near 70 (because of y=2x+30) . The output can be seen in the following:

Here weโ€™re getting returned the value 70.05354 which is pretty close to 70.0, so that our model is working as expected.

Getting Model Insights, Retrieving Weights And Bias

Weโ€™re able to get more model insights by taking a look at the weight and the bias which is determined for the first layer:

print("These are the layer variables: {}".format(model.layers[0].get_weights()))

As expected weโ€™re getting returned two parameter for our first and only layer in the model:

The two parameters corresponds to the two variables we have in the model:

  • Weight
  • Bias

For the weight the value which is determined is near the target value of 2 and for the bias the value which is determined is near the target value of 30 (according to our linear regression formula: y = 2x + 30).

Machine Learning Full Course - Learn Machine Learning

Machine Learning Full Course - Learn Machine Learning

This complete Machine Learning full course video covers all the topics that you need to know to become a master in the field of Machine Learning.

Machine Learning Full Course | Learn Machine Learning | Machine Learning Tutorial

It covers all the basics of Machine Learning (01:46), the different types of Machine Learning (18:32), and the various applications of Machine Learning used in different industries (04:54:48).This video will help you learn different Machine Learning algorithms in Python. Linear Regression, Logistic Regression (23:38), K Means Clustering (01:26:20), Decision Tree (02:15:15), and Support Vector Machines (03:48:31) are some of the important algorithms you will understand with a hands-on demo. Finally, you will see the essential skills required to become a Machine Learning Engineer (04:59:46) and come across a few important Machine Learning interview questions (05:09:03). Now, let's get started with Machine Learning.

Below topics are explained in this Machine Learning course for beginners:

  1. Basics of Machine Learning - 01:46

  2. Why Machine Learning - 09:18

  3. What is Machine Learning - 13:25

  4. Types of Machine Learning - 18:32

  5. Supervised Learning - 18:44

  6. Reinforcement Learning - 21:06

  7. Supervised VS Unsupervised - 22:26

  8. Linear Regression - 23:38

  9. Introduction to Machine Learning - 25:08

  10. Application of Linear Regression - 26:40

  11. Understanding Linear Regression - 27:19

  12. Regression Equation - 28:00

  13. Multiple Linear Regression - 35:57

  14. Logistic Regression - 55:45

  15. What is Logistic Regression - 56:04

  16. What is Linear Regression - 59:35

  17. Comparing Linear & Logistic Regression - 01:05:28

  18. What is K-Means Clustering - 01:26:20

  19. How does K-Means Clustering work - 01:38:00

  20. What is Decision Tree - 02:15:15

  21. How does Decision Tree work - 02:25:15ย 

  22. Random Forest Tutorial - 02:39:56

  23. Why Random Forest - 02:41:52

  24. What is Random Forest - 02:43:21

  25. How does Decision Tree work- 02:52:02

  26. K-Nearest Neighbors Algorithm Tutorial - 03:22:02

  27. Why KNN - 03:24:11

  28. What is KNN - 03:24:24

  29. How do we choose 'K' - 03:25:38

  30. When do we use KNN - 03:27:37

  31. Applications of Support Vector Machine - 03:48:31

  32. Why Support Vector Machine - 03:48:55

  33. What Support Vector Machine - 03:50:34

  34. Advantages of Support Vector Machine - 03:54:54

  35. What is Naive Bayes - 04:13:06

  36. Where is Naive Bayes used - 04:17:45

  37. Top 10 Application of Machine Learning - 04:54:48

  38. How to become a Machine Learning Engineerย - 04:59:46

  39. Machine Learning Interview Questions - 05:09:03

Machine Learning Tutorial - Learn Machine Learning - Intellipaat

Machine Learning Tutorial - Learn Machine Learning - Intellipaat

This Machine Learning tutorial for beginners will enable you to learn Machine Learning algorithms with python examples. Become a pro in Machine Learning.

Mastering the Machine Learning Course would easily develop one's career. This is the reason why studying Machine Learning Tutorial becomes so important in the career of a particular student.
Making a part of the machine learning course would enact and studying the Machine Learning Tutorial would make one carve out a new niche.

Top Machine Learning Framework: 5 Machine Learning Frameworks of 2019

Top Machine Learning Framework: 5 Machine Learning Frameworks of 2019

Machine Learning (ML) is one of the fastest-growing technologies today. ML has a lot of frameworks to build a successful app, and so as a developer, you might be getting confused about using the right framework. Herein we have curated top 5...

Machine Learning (ML) is one of the fastest-growing technologies today. ML has a lot of frameworks to build a successful app, and so as a developer, you might be getting confused about using the right framework. Herein we have curated top 5 machine learning frameworks that are cutting edge technology in your hands.

Through the machine learning frameworks, mobile phones and tablets are getting powerful enough to run the software that can learn and react in real-time. It is a complex discipline. But the implementation of ML models is far less daunting and difficult than it used to be. Now, it automatically improves the performance with the pace of time, interactions, and experiences, and the most important acquisition of useful data pertaining to the tasks allocated.

As we know that ML is considered as a subset of Artificial Intelligence (AI). The scientific study of statistical models and algorithms help a computing system to accomplish designated tasks efficiently. Now, as a mobile app developer, when you are planning to choose machine learning frameworks you must keep the following things in mind.

The framework should be performance-oriented
The grasping and coding should be quick
It allows to distribute the computational process, the framework must have parallelization
It should consist of a facility to create models and provide a developer-friendly tool
Letโ€™s learn about the top five machine learning frameworks to make the right choice for your next ML application development project. Before we dive deeper into these mentioned frameworks, know the different types of ML frameworks that are available on the web. Here are some ML frameworks:

Mathematical oriented
Neural networks-based
Linear algebra tools
Statistical tools
Now, letโ€™s have an insight into ML frameworks that will help you in selecting the right framework for your ML application.

Donโ€™t Miss Out on These 5 Machine Learning Frameworks of 2019
#1 TensorFlow
TensorFlow is an open-source software library for data-based programming across multiple tasks. The framework is based on computational graphs which is essentially a network of codes. Each node represents a mathematical operation that runs some function as simple or as complex as multivariate analysis. This framework is said to be best among all the ML libraries as it supports regressions, classifications, and neural networks like complicated tasks and algorithms.

machine learning frameworks
This machine learning library demands additional efforts while learning TensorFlow Python framework. Your job becomes easy in the n-dimensional array of the framework when you have grasped the Python frameworks and libraries.

The benefits of this framework are flexibility. TensorFlow allows non-automatic migration to newer versions. It runs on the GPU, CPU, servers, desktops, and mobile devices. It provides auto differentiation and performance. There are a few goliaths like Airbus, Twitter, IBM, who have innovatively used the TensorFlow frameworks.

#2 FireBase ML Kit
Firebase machine learning framework is a library that allows effortless, minimal code, with highly accurate, pre-trained deep models. We at Space-O Technologies use this machine learning technology for image classification and object detection. The Firebase framework offers models both locally and on the Google Cloud.

machine learning frameworks
This is one of our ML tutorials to make you understand the Firebase frameworks. First of all, we collected photos of empty glass, half watered glass, full watered glass, and targeted into the machine learning algorithms. This helped the machine to search and analyze according to the nature, behavior, and patterns of the object placed in front of it.

The first photo that we targeted through machine learning algorithms was to recognize an empty glass. Thus, the app did its analysis and search for the correct answer, we provided it with certain empty glass images prior to the experiment.
The other photo that we targeted was a half water glass. The core of the machine learning app is to assemble data and to manage it as per its analysis. It was able to recognize the image accurately because of the little bits and pieces of the glass given to it beforehand.
The last one is a full glass recognition image.
Note: For correct recognition, there has to be 1 label that carries at least 100 images of a particular object.

#3 CAFFE (Convolutional Architecture for Fast Feature Embedding)
CAFFE framework is the fastest way to apply deep neural networks. It is the best machine learning framework known for its model-Zoo a pre-trained ML model that is capable of performing a great variety of tasks. Image classification, machine vision, recommender system are some of the tasks performed easily through this ML library.

machine learning frameworks
This framework is majorly written in CPP. It can run on multiple hardware and can switch between CPU and GPU with the use of a single flag. It has systematically organized the structure of Mat lab and python interface.

Now, if you have to make a machine learning app development, then it is mainly used in academic research projects and to design startups prototypes. It is the aptest machine learning technology for research experiments and industry deployment. At a time this framework can manage 60 million pictures every day with a solitary Nvidia K40 GPU.

#4 Apache Spark
The Apache Spark machine learning is a cluster-computing framework written in different languages like Java, Scala, R, and Python. Sparkโ€™s machine learning library, MLlib is considered as foundational for the Sparkโ€™s success. Building MLlib on top of Spark makes it possible to tackle the distinct needs of a single tool instead of many disjointed ones.

machine learning frameworks
The advantages of such ML library lower learning curves, less complex development and production environments, which ultimately results in a shorter time to deliver high-performing models. The key benefit of MLlib is that it allows data scientists to solve multiple data problems in addition to their machine learning problems.

It can easily solve graph computations (via GraphX), streaming (real-time calculations), and real-time interactive query processing with Spark SQL and DataFrames. The data professionals can focus on solving the data problems instead of learning and maintaining a different tool for each scenario.

#5 Scikit-Learn
Scikit-learn is said to be one of the greatest feats of Python community. This machine learning framework efficiently handles data mining and supports multiple practical tasks. It is built on foundations like SciPy, Numpy, and matplotlib. This framework is known for supervised & unsupervised learning algorithms as well as cross-validation. The Scikit learn is largely written in Python with some core algorithms in Cython to achieve performance.

machine learning frameworks
The machine learning framework can work on multiple tasks without compromising on speed. There are some remarkable machine learning apps using this framework like Spotify, Evernote, AWeber, Inria.

With the help of machine learning to build iOS apps, Android apps powered by ML have become quite an easy process. With this emerging technology trend varieties of available data, computational processing has become cheaper and more powerful, and affordable data storage. So being an app developer or having an idea for machine learning apps should definitely dive into the niche.

Conclusion
Still have any query or confusion regarding ML frameworks, machine learning app development guide, the difference between Artificial Intelligence and machine learning, ML algorithms from scratch, how this technology is helpful for your business? Just fill our contact us form. Our sales representatives will get back to you shortly and resolve your queries. The consultation is absolutely free of cost.

Author Bio: This blog is written with the help of Jigar Mistry, who has over 13 years of experience in the web and mobile app development industry. He has guided to develop over 200 mobile apps and has special expertise in different mobile app categories like Uber like apps, Health and Fitness apps, On-Demand apps and Machine Learning apps. So, we took his help to write this complete guide on machine learning technology and machine app development areas.