The Machine Learning Crash Course – Part 1👏👏👏

The Machine Learning Crash Course – Part 1👏👏👏

The Machine Learning Crash Course - Welcome to the first episode of the Machine Learning Crash Course For Beginners.A self-study guide for aspiring machine learning practitioners...🌟🌟🌟🌟🌟

The Machine Learning Crash Course - Welcome to the first episode of the Machine Learning Crash Course For Beginners.A self-study guide for aspiring machine learning practitioners...

In this series you’ll learn everything you need to get started with Machine Learning from scratch. No prior knowledge is required. You’ll learn basic concepts and practical applications from start to finish.

In this first episode we’ll start to clarify what Machine Learning exactly is, how it is linked to terms like Artificial Intelligence and Deep Learning and what different types of Machine Learning are available.

We’ll also take a look at Google Colaboratory (Colab) which is a Python-based Machine Learning development environment in the cloud.

Artificial Intelligence, Machine Learning, and Deep Learning explained

In the context of Machine Learning people often talk about Artificial Intelligence and Deep Learning. So let’s see how these terminologies relate to each other.

The most comprehensive term is Artificial Intelligence. Artificial Intelligence is a field in computer science that aims to make computers achieve a human-style intelligence.

Machine Learning is then located within the area of Artificial Intelligence and describes a set of techniques which are used to train a computer to perform a certain task without providing the exact algorithm which is needed to solve that task.

Deep Learning is a sub-field of Machine Learning. In Deep Learning multi-layered neural networks are used to perform machine learning tasks. This is at the same time the reason why it is called “Deep” Learning. The term deep refers to the number of layers in the neural network through which the data is transformed.

Take a look at the following illustration to see how Artificial Intelligence, Machine Learning, and Deep Learning relate to each other:

Supervised vs. Unsupervised Learning

Let’s now take a closer took at Deep Learning. In general Deep Learning can again be split into two sub-fields: Supervised vs. Unsupervised Learning. The following illustration is showing that fact:

In general Supervised Learning is describing the learning process when the computer knows what to learn. That means that the input data which is provided to train a neural network is labeled. With the labels provided the computer knows what the correct answer would be.

A typical example of that type of learning is an image recognition task. For training the neural network images are provided which are labeled, so that the neural network knows what is shown on the images. E.g. you can think of images showing cats and dogs and those images are labeled with “cat” or “dog. Providing the set of training data makes it possible to train the neural network and afterwards provide images showing cats or dogs without label and letting the neural network recognise what is shown on the image by providing the answer “cat” or “dog”.

In contrast in Unsupervised Learning it is unkown what the computer needs to learn. That means that you only have input data without corresponding labels. In that case the Machine Learning algorithm needs to figure out how to interpret and how to structure the data by itself. There are two main tasks for Unsupervised Learning:

  • Unsupervised Learning For Clustering: Clustering means that the algorithm tries to group together the input data in categories. Example: red, blue, green etc. The category is then the label which is determined for the provided input data.
  • Unsupervised Learning For Association: Association means that the algorithm tries to determine rules which are describing large portions of the data. E.g. poeple that smoke tend to get cancer.
Machine Learning vs. Traditional Programming

Now that we’ve taken a first look into concepts of Machine Learning you should have a first understanding of what Machine Learning / Deep Learning is. Let’s compare the general approach of Machine Learning to the approach of traditional programming to get a better understanding of the general concept.

Take a look at the following picture first:

What you can see here is the direct comparison of the approach of traditional programming and the approach which is taken which Machine Learning.

In traditional programming we’re used to provide input data and rules in order to implement the program. Usually rules are being used to implement an algorithm which is then processing input data which leads to a certain output.

In contrast to that the approach in Machine Learning is a little bit different. In Machine Learning we’re providing input data together with the related output first. Just think of the example with images (input) and the corresponding image labels (output). Based on what is being provided we’re training a neural network which then delivers the rules. With that trained neural network we’re then able to apple the neural network on new input data and it will be able to prodict the output with a certain accuracy.

Practical Machine Learning

Python And TensorFlow

Next, let’s start with the first practical part. In order to write code for Machine Learning we’ll use a framework which is called TensorFlow. TensorFlow is developed and provided by Google and is an end-to-end open source machine learning platform which is available for many programming languages.

For the first practical examples in this Machine Learning Crash Course we’ll be using TensorFlow together with the Python programming language. There are three main reasons for this choice:

  • Python is a programming language which is easy to learn and easy to read. Even if you’re not a Python expert you’ll be able to follow along the examples.
  • Python offers libraries which makes it easy to deal with large sets of data. This is the prerequisite for any machine learning exercise.
  • With Google’s Colab cloud environment we’re able to use a fully prepared Python Machine Learning development environment in the cloud which is ready-to-use without needed to setup anything on your local machine first.

If you’d like to get an overview of the TensorFlow platform first you can take a look at the project’s website at https://tensorflow.org:

https://www.python.org/ where you can find additional information on all aspects of the language.

Google’s Colab – A Machine Learning Development Environment In The Cloud

So now, that we know that Python and TensorFlow should be used when implementing our first Machine Learning examples we need to setup a development environment first.

Of course you can do that on your local system in many different ways:

  • You can download and install Python and TensorFlow and use any code editor you like to write your Machine Learning code
  • You can setup Jupyter Nodebook (https://jupyter.org/) "https://jupyter.org/)") in order to host your development environment on your local machine and access the Jupyter Nodebook web application in the browser.
  • You can use use Google Colaboratory (Colab) which is a Jupyter Notebook-based Python development environment which is provided in the Google Cloud. Colab provides everything what is needed to get started with Machine Learning. TensorFlow is already pre-installed and as Colab is free to use you just need to sign up with your Google Account to be able to access Colab.

For the following exercises we’ll be using Google’s Colab. This is the most easiest way to get a access to a full-featured Python-based Machine Learning development environment. Let’s take a closed look at the most important features Colab has to offer.

Go to https://colab.research.google.com/ and login with your Google account. Afterwards you will be presented with the with the following dialog which let’s you choose to

  • open any of the provided example notebooks
  • open existing notebooks from Google Drive or Github
  • open existing notebooks by selecting and uploading a file from your local machine
  • creating a new and empty Python 3 notebook by hitting the link NEW PYTHON 3 NOTEBOOK at the bottom:

In order to make ourselves familiar with some of the basic Colab functionalities let’s create such a new and empty Python 3 notebook:

The notebook editor is organised in cells. A notebook cell can contain two different types of content:

  • Code
  • Text

By default the notebook already contains one code cell, so that we can start typing in Python code like you can see in the following screenshot:

The code within the cell can then be executed by either hitting the play button located on the left side of the cell or hitting CTRL+RETURN. The code is being executed and the output which is generated by the code execution is displayed below:

By using the buttons + CODE and + TEXT you can add more cells to your notebook. For example add a new text cell to add some explanatory text:

As you can see the cell text content can also be formatted by using the corresponding icons are typing in markdown code. As long as you’re in editing mode the cell is displayed in a split view. The left side of the text cell is containing the markdown text editor and the right side of the cell is showing the resulting output.

If you’re leaving the edit mode of the text cell (e.g. by clicking just somewhere outside of the cell) the compiled text output is displayed in the notebook:

Let’s add another code cell next and include the Python code which is needed to iterate over a simple list of value. For each iteration the actual value is outputted:

Now you can see that the output of the cell is containing one line per iteration listing the respective values.

If you need additional functionality which is not part of the Python core language you can use additional. The most common libraries used in Machine Learning are already pre-installed, e.g. the NumPy library to manage arrays more easily. Let’s try it out by adding another code cell to our notebook:

Here we’re first making sure that the NumPy library is imported, so that we’re able to use it. Next we’re creating a new array a by using the NumPy array function. Initially this array is consisting of just two elements and then extended by two more elements (by using the append function). Finally the array is printed out. Once the code cell is being executed you should be able to see the corresponding output right below the code cell.

The NumPy library can also be used to do some basic mathematical operations on array values like you can see in the following:

Here we’re first initialising a array with numeric values. Then we’re printing out the maximum value of that array (by using function max), the average of all values included in that array (by using average function) and the index of the maximum value in the array (by using the argmax function).

A use case for which NumPy is often used in Machine Learning is creating Arrays of a certain dimension and initialising this array which random numbers:

You can also access the shape of a NumPy array and print it out:

The Colab environment is not limited to only execute code. Colab is also a virtual machine, so that you can execute commands that you also would also use on the command line. Take a look at the following example, another code cell is used to execute a few command:

What you can see here is that you need to prefix the line which contains the command with an exclamation mark.

Colab gives you also access to code snippets. To access the list of snippets just expand the left-side menu by clicking on the arrow and you should be able to see the list of available snippets:

You can use the search field to input text and search for matchings in the list of available snippets. Double-click on a specific entry to insert the code snippet as a new code cell to the notebook. In the following you can see the result of inserting and executing the snippet “Visualization: Histogram in Altair”:

Conclusion

In this first episode of the Machine Learning Crash Course you’ve learned about some basic terminologies and we’ve started to make ourselves familiar with Colab, so that we’re able to use this Python Cloud development environment to start with a first practical Machine Learning project in the next part of this tutorial.

Machine Learning Full Course - Learn Machine Learning

Machine Learning Full Course - Learn Machine Learning

This complete Machine Learning full course video covers all the topics that you need to know to become a master in the field of Machine Learning.

Machine Learning Full Course | Learn Machine Learning | Machine Learning Tutorial

It covers all the basics of Machine Learning (01:46), the different types of Machine Learning (18:32), and the various applications of Machine Learning used in different industries (04:54:48).This video will help you learn different Machine Learning algorithms in Python. Linear Regression, Logistic Regression (23:38), K Means Clustering (01:26:20), Decision Tree (02:15:15), and Support Vector Machines (03:48:31) are some of the important algorithms you will understand with a hands-on demo. Finally, you will see the essential skills required to become a Machine Learning Engineer (04:59:46) and come across a few important Machine Learning interview questions (05:09:03). Now, let's get started with Machine Learning.

Below topics are explained in this Machine Learning course for beginners:

  1. Basics of Machine Learning - 01:46

  2. Why Machine Learning - 09:18

  3. What is Machine Learning - 13:25

  4. Types of Machine Learning - 18:32

  5. Supervised Learning - 18:44

  6. Reinforcement Learning - 21:06

  7. Supervised VS Unsupervised - 22:26

  8. Linear Regression - 23:38

  9. Introduction to Machine Learning - 25:08

  10. Application of Linear Regression - 26:40

  11. Understanding Linear Regression - 27:19

  12. Regression Equation - 28:00

  13. Multiple Linear Regression - 35:57

  14. Logistic Regression - 55:45

  15. What is Logistic Regression - 56:04

  16. What is Linear Regression - 59:35

  17. Comparing Linear & Logistic Regression - 01:05:28

  18. What is K-Means Clustering - 01:26:20

  19. How does K-Means Clustering work - 01:38:00

  20. What is Decision Tree - 02:15:15

  21. How does Decision Tree work - 02:25:15 

  22. Random Forest Tutorial - 02:39:56

  23. Why Random Forest - 02:41:52

  24. What is Random Forest - 02:43:21

  25. How does Decision Tree work- 02:52:02

  26. K-Nearest Neighbors Algorithm Tutorial - 03:22:02

  27. Why KNN - 03:24:11

  28. What is KNN - 03:24:24

  29. How do we choose 'K' - 03:25:38

  30. When do we use KNN - 03:27:37

  31. Applications of Support Vector Machine - 03:48:31

  32. Why Support Vector Machine - 03:48:55

  33. What Support Vector Machine - 03:50:34

  34. Advantages of Support Vector Machine - 03:54:54

  35. What is Naive Bayes - 04:13:06

  36. Where is Naive Bayes used - 04:17:45

  37. Top 10 Application of Machine Learning - 04:54:48

  38. How to become a Machine Learning Engineer - 04:59:46

  39. Machine Learning Interview Questions - 05:09:03

Machine Learning | Machine Learning Guide for Beginners

Machine Learning | Machine Learning Guide for Beginners

Machine learning problems can generally be divided into three types. Classification and regression, which are known as supervised learning, and unsupervised learning which in the context of machine learning applications often refers to clustering.

Machine learning problems can generally be divided into three types. Classification and regression, which are known as supervised learning, and unsupervised learning which in the context of machine learning applications often refers to clustering.

In the following article, I am going to give a brief introduction to each of these three problems and will include a walkthrough in the popular python library scikit-learn.

Before I start I’ll give a brief explanation for the meaning behind the terms supervised and unsupervised learning.

Supervised Learning: In supervised learning, you have a known set of inputs (features) and a known set of outputs (labels). Traditionally these are known as X and y. The goal of the algorithm is to learn the mapping function that maps the input to the output. So that when given new examples of X the machine can correctly predict the corresponding y labels.

Unsupervised Learning: In unsupervised learning, you only have a set of inputs (X) and no corresponding labels (y). The goal of the algorithm is to find previously unknown patterns in the data. Quite often these algorithms are used to find meaningful clusters of similar samples of X so in effect finding the categories intrinsic to the data.

Classification

In classification, the outputs (y) are categories. These can be binary, for example, if we were classifying spam email vs not spam email. They can also be multiple categories such as classifying species of flowers, this is known as multiclass classification.

Let’s walk through a simple example of classification using scikit-learn. If you don’t already have this installed it can be installed either via pip or conda as outlined here.

Scikit-learn has a number of datasets that can be directly accessed via the library. For ease in this article, I will be using these example datasets throughout. To illustrate classification I will use the wine dataset which is a multiclass classification problem. In the dataset, the inputs (X) consist of 13 features relating to various properties of each wine type. The known outputs (y) are wine types which in the dataset have been given a number 0, 1 or 2.

The imports I am using for all the code in this article are shown below.

import pandas as pd
import numpy as np
from sklearn.datasets import load_wine
from sklearn.datasets import load_boston
from sklearn.model_selection import train_test_split
from sklearn import preprocessing
from sklearn.metrics import f1_score
from sklearn.metrics import mean_squared_error
from math import sqrt
from sklearn.neighbors import KNeighborsClassifier
from sklearn.svm import SVC, LinearSVC, NuSVC
from sklearn.tree import DecisionTreeClassifier
from sklearn.ensemble import RandomForestClassifier, AdaBoostClassifier, GradientBoostingClassifier
from sklearn.discriminant_analysis import LinearDiscriminantAnalysis
from sklearn.discriminant_analysis import QuadraticDiscriminantAnalysis
from sklearn import linear_model
from sklearn.linear_model import ElasticNetCV
from sklearn.svm import SVR
from sklearn.cluster import KMeans
from yellowbrick.cluster import KElbowVisualizer
from yellowbrick.cluster import SilhouetteVisualizer

In the below code I am downloading the data and converting to a pandas data frame.

wine = load_wine()
wine_df = pd.DataFrame(wine.data, columns=wine.feature_names)
wine_df['TARGET'] = pd.Series(wine.target)

The next stage in a supervised learning problem is to split the data into test and train sets. The train set can be used by the algorithm to learn the mapping between inputs and outputs, and then the reserved test set can be used to evaluate how well the model has learned this mapping. In the below code I am using the scikit-learn model_selection function train_test_split to do this.

X_w = wine_df.drop(['TARGET'], axis=1)
y_w = wine_df['TARGET']
X_train_w, X_test_w, y_train_w, y_test_w = train_test_split(X_w, y_w, test_size=0.2)

In the next step, we need to choose the algorithm that will be best suited to learn the mapping in your chosen dataset. In scikit-learn there are many different algorithms to choose from, all of which use different functions and methods to learn the mapping, you can view the full list here.

To determine the best model I am running the following code. I am training the model using a selection of algorithms and obtaining the F1-score for each one. The F1 score is a good indicator of the overall accuracy of a classifier. I have written a detailed description of the various metrics that can be used to evaluate a classifier here.

classifiers = [
    KNeighborsClassifier(3),
    SVC(kernel="rbf", C=0.025, probability=True),
    NuSVC(probability=True),
    DecisionTreeClassifier(),
    RandomForestClassifier(),
    AdaBoostClassifier(),
    GradientBoostingClassifier()
    ]
for classifier in classifiers:
    model = classifier
    model.fit(X_train_w, y_train_w)  
    y_pred_w = model.predict(X_test_w)
    print(classifier)
    print("model score: %.3f" % f1_score(y_test_w, y_pred_w, average='weighted'))

A perfect F1 score would be 1.0, therefore, the closer the number is to 1.0 the better the model performance. The results above suggest that the Random Forest Classifier is the best model for this dataset.

Regression

In regression, the outputs (y) are continuous values rather than categories. An example of regression would be predicting how many sales a store may make next month, or what the future price of your house might be.

Again to illustrate regression I will use a dataset from scikit-learn known as the boston housing dataset. This consists of 13 features (X) which are various properties of a house such as the number of rooms, the age and crime rate for the location. The output (y) is the price of the house.

I am loading the data using the code below and splitting it into test and train sets using the same method I used for the wine dataset.

boston = load_boston()
boston_df = pd.DataFrame(boston.data, columns=boston.feature_names)
boston_df['TARGET'] = pd.Series(boston.target)
X_b = boston_df.drop(['TARGET'], axis=1)
y_b = boston_df['TARGET']
X_train_b, X_test_b, y_train_b, y_test_b = train_test_split(X_b, y_b, test_size=0.2)

We can use this cheat sheet to see the available algorithms suited to regression problems in scikit-learn. We will use similar code to the classification problem to loop through a selection and print out the scores for each.

There are a number of different metrics used to evaluate regression models. These are all essentially error metrics and measure the difference between the actual and predicted values achieved by the model. I have used the root mean squared error (RMSE). For this metric, the closer to zero the value is the better the performance of the model. This article gives a really good explanation of error metrics for regression problems.

regressors = [
    linear_model.Lasso(alpha=0.1),
    linear_model.LinearRegression(),
    ElasticNetCV(alphas=None, copy_X=True, cv=5, eps=0.001, fit_intercept=True,
       l1_ratio=0.5, max_iter=1000, n_alphas=100, n_jobs=None,
       normalize=False, positive=False, precompute='auto', random_state=0,
       selection='cyclic', tol=0.0001, verbose=0),
    SVR(C=1.0, cache_size=200, coef0=0.0, degree=3, epsilon=0.1,
    gamma='auto_deprecated', kernel='rbf', max_iter=-1, shrinking=True,
    tol=0.001, verbose=False),
    linear_model.Ridge(alpha=.5)                
    ]
for regressor in regressors:
    model = regressor
    model.fit(X_train_b, y_train_b)  
    y_pred_b = model.predict(X_test_b)
    print(regressor)
    print("mean squared error: %.3f" % sqrt(mean_squared_error(y_test_b, y_pred_b)))


The RMSE score suggests that either the linear regression and ridge regression algorithms perform best for this dataset.

Unsupervised learning

There are a number of different types of unsupervised learning but for simplicity here I am going to focus on the clustering methods. There are many different algorithms for clustering all of which use slightly different techniques to find clusters of inputs.

Probably one of the most widely used methods is Kmeans. This algorithm performs an iterative process whereby a specified number of randomly generated means are initiated. A distance metric, Euclidean distance is calculated for each data point from the centroids, thus creating clusters of similar values. The centroid of each cluster then becomes the new mean and this process is repeated until the optimum result has been achieved.

Let’s use the wine dataset we used in the classification task, with the y labels removed, and see how well the k-means algorithm can identify the wine types from the inputs.

As we are only using the inputs for this model I am splitting the data into test and train using a slightly different method.

np.random.seed(0)
msk = np.random.rand(len(X_w)) < 0.8
train_w = X_w[msk]
test_w = X_w[~msk]

As Kmeans is reliant on the distance metric to determine the clusters it is usually necessary to perform feature scaling (ensuring that all features have the same scale) before training the model. In the below code I am using the MinMaxScaler to scale the features so that all values fall between 0 and 1.

x = train_w.values
min_max_scaler = preprocessing.MinMaxScaler()
x_scaled = min_max_scaler.fit_transform(x)
X_scaled = pd.DataFrame(x_scaled,columns=train_w.columns)

With K-means you have to specify the number of clusters the algorithm should use. So one of the first steps is to identify the optimum number of clusters. This is achieved by iterating through a number of values of k and plotting the results on a chart. This is known as the Elbow method as it typically produces a plot with a curve that looks a little like the curve of your elbow. The yellowbrick library (which is a great library for visualising scikit-learn models and can be pip installed) has a really nice plot for this. The code below produces this visualisation.

model = KMeans()
visualizer = KElbowVisualizer(model, k=(1,8))
visualizer.fit(X_scaled)       
visualizer.show()

Ordinarily, we wouldn’t already know how many categories we have in a dataset where we are using a clustering technique. However, in this case, we know that there are three wine types in the data — the curve has correctly selected three as the optimum number of clusters to use in the model.

The next step is to initialise the K-means algorithm and fit the model to the training data and evaluate how effectively the algorithm has clustered the data.

One method used for this is known as the silhouette score. This measures the consistency of values within the clusters. Or in other words how similar to each other the values in each cluster are, and how much separation there is between the clusters. The silhouette score is calculated for each value and will range from -1 to +1. These values are then plotted to form a silhouette plot. Again yellowbrick provides a simple way to construct this type of plot. The code below creates this visualisation for the wine dataset.

model = KMeans(3, random_state=42)
visualizer = SilhouetteVisualizer(model, colors='yellowbrick')
visualizer.fit(X_scaled)      
visualizer.show()

A silhouette plot can be interpreted in the following way:

  • The closer the mean score (which is the red dotted line in the above) is to +1 the better matched the data points are within the cluster.
  • Data points with a score of 0 are very close to the decision boundary for another cluster (so the separation is low).
  • Negative values indicate that the data points may have been assigned to the wrong cluster.
  • The width of each cluster should be reasonably uniform if they aren’t then the incorrect value of k may have been used.

The plot for the wine data set above shows that cluster 0 may not be as consistent as the others due to most data points being below the average score and a few data points having a score below 0.

Silhouette scores can be particularly useful in comparing one algorithm against another or different values of k.

In this post, I wanted to give a brief introduction to each of the three types of machine learning. There are many other steps involved in all of these processes including feature engineering, data processing and hyperparameter optimisation to determine both the best data preprocessing techniques and the best models to use.

Thanks for reading!