In my previous post about ensemble learning, I have explained what is ensemble learning, how it relates with Bias and Variance in machine learning and what are the simple techniques of ensemble learning. If you haven’t read the post, please refer here.

In this post I will cover ensemble learning types, advanced ensemble learning methods — Bagging, Boosting, Stacking and Blending with code samples. At the end I will explain some pros and cons of using ensemble learning.

Ensemble Learning Types

Ensemble learning methods can be categorized into two groups:

1. Sequential Ensemble Methods

In this method base learners are dependent on the results from previous base learners. Every subsequent base model corrects the prediction made by its predecessor fixing the errors in it. Hence the overall performance can be increased via improving the weight of previous labels.

2. Parallel Ensemble Methods

In this method there is no dependency between the base learners and all base learners execute in parallel and the results of all base models are combined in the end (using averaging for regression and voting for classification problems).

Parallel Ensemble methods are divided in two categories-

**1. Homogeneous Parallel Ensemble Methods- **In this method a single machine learning algorithm is used as a base learner.

2. Heterogeneous Parallel Ensemble Methods- In this method multiple machine learning algorithms are used as base learners.

Advanced Ensemble Techniques

Bagging

Bagging or Bootstrap Aggregation is a parallel ensemble learning technique to reduce the variance in the final prediction.

The Bagging process is very similar to averaging, only difference is that bagging uses random sub-samples of the original dataset to train same/multiple models and then combines the prediction, whereas in averaging the same dataset is used to train models. Hence the technique is called Bootstrap Aggregation as it combines both Bootstrapping (or Sampling of data) and Aggregation to form an ensemble model.

#artificial-intelligence #ensemble-learning #ensemble #data-science #machine-learning