Bagging on Low Variance Models

Bagging on Low Variance Models

Bagging on Low Variance Models. A curious case of bagging on simple linear regression

Bagging (also known as bootstrap aggregation) is a technique in which we take multiple samples repeatedly with replacement according to uniform probability distribution and fit a model on it. It combines multiple predictions to give a better prediction by majority vote or taking the aggregate of the predictions. This technique is effective on models which tend to overfit on the dataset (high variance models). Bagging reduces the variance without making the predictions biased. This technique acts as a base to many ensemble techniques so understanding the intuition behind it is crucial.

If this technique is so good, why do we use it only on models which show high variance? What happens when we use it with models which have low variance? Let us try to understand the underlying issue with the help of a demonstration.

We will be using bagging on decision tree to prove that bagging improves the accuracy for high variance models and compare it to bagging on simple linear regression which is biased depending on the dataset. Simple linear regression is biased when the predictor is not perfectly correlated to the target variable.

Bias and Variance

We will be talking about Bias and Variance throughout the article so let us get an idea of what it is first.


High bias refers to the oversimplification of the model. i.e. When we are unable to capture the true relation of the data. Our objective of creating a model is to capture the true nature of the data and predict based on the trend, which makes high bias an undesired phenomenon.

bias-variance-tradeoff bagging ensemble-learning linear-regression machine-learning

Bootstrap 5 Complete Course with Examples

Bootstrap 5 Tutorial - Bootstrap 5 Crash Course for Beginners

Nest.JS Tutorial for Beginners

Hello Vue 3: A First Look at Vue 3 and the Composition API

Building a simple Applications with Vue 3

Deno Crash Course: Explore Deno and Create a full REST API with Deno

How to Build a Real-time Chat App with Deno and WebSockets

Convert HTML to Markdown Online

HTML entity encoder decoder Online

Linear Regression VS Logistic Regression (MACHINE LEARNING)

Linear Regression VS Logistic Regression (MACHINE LEARNING). Linear Regression and Logistic Regression are two algorithms of machine learning and these are mostly used in the data science field.

The Bias-Variance Tradeoff

Here, we will go over what Bias error and Variance error are, sources of these errors and how you can work to reduce these errors in your model.

Regression: Linear Regression

Machine learning algorithms are not your regular algorithms that we may be used to because they are often described by a combination of some complex statistics and mathematics.

What is Supervised Machine Learning

What is neuron analysis of a machine? Learn machine learning by designing Robotics algorithm. Click here for best machine learning course models with AI

Pros and Cons of Machine Learning Language

AI, Machine learning, as its title defines, is involved as a process to make the machine operate a task automatically to know more join CETPA