Feature Selection Methods in Machine Learning

Feature Selection Methods in Machine Learning

Wrapper: Search for well-performing subsets of features. RFE. Filter: Select subsets of features based on their relationship with the target. Feature Importance Methods. Intrinsic: Algorithms that perform automatic feature selection during training.

Introduction

When do you say a model is good? When a model performs well on unseen data then we say its a good model. As a Data Scientist, we perform various operations in order to make a good Machine Learning model. The operations include data pre-processing (dealing with NA’s and outliers, column type conversions, dimensionality reduction, normalization etc), exploratory data analysis (EDA), hyperparameter tuning/optimization (the process of finding the best set of hyper-parameters of the ML algorithm that delivers the best performance), feature selection etc.

“Garbage in, Garbage out.”

If data fed into an ML model is of poor quality, the model will be of poor quality

Below articles are related to hyperparameter tuning techniques, open-source frameworks for hyperparameter tuning and data leakage in hyperparameter tuning.

Feature Selection

Feature Selection is the process of selecting the best subset from the existing set of features that contribute more in predicting the output. This helps to improve the performance of the model (i.e removes redundant or/and irrelevant features that are carrying noise and decreasing the accuracy of the model). One of the major advantages of feature selection is it rescues the model from the high risk of overfitting when we have datasets with high dimensionality. By reducing the number of features, it actually reduces the training time of the ML algorithm i.e. computational cost involved.

There are 3 main feature selection techniques

  1. Filter methods
  2. Embedded methods
  3. Wrapper methods
import numpy as np
import pandas as pd 
import matplotlib.pyplot as plt
import seaborn as sns
sns.set(style=”whitegrid”) 
import warnings
warnings.filterwarnings(‘ignore’) 
## load Iris dataset 
sklearn.datasets import load_iris 
## create input and output features
feature_names = load_iris().feature_names
X_data = pd.DataFrame(load_iris().data, columns=feature_names)
y_data = load_iris().target

data-science artificial-intelligence python ai machine-learning

Bootstrap 5 Complete Course with Examples

Bootstrap 5 Tutorial - Bootstrap 5 Crash Course for Beginners

Nest.JS Tutorial for Beginners

Hello Vue 3: A First Look at Vue 3 and the Composition API

Building a simple Applications with Vue 3

Deno Crash Course: Explore Deno and Create a full REST API with Deno

How to Build a Real-time Chat App with Deno and WebSockets

Convert HTML to Markdown Online

HTML entity encoder decoder Online

Data Science Projects | Data Science | Machine Learning | Python

Practice your skills in Data Science with Python, by learning and then trying all these hands-on, interactive projects, that I have posted for you.

Data Science Projects | Data Science | Machine Learning | Python

Practice your skills in Data Science with Python, by learning and then trying all these hands-on, interactive projects, that I have posted for you.

Data Science Projects | Data Science | Machine Learning | Python

Practice your skills in Data Science with Python, by learning and then trying all these hands-on, interactive projects, that I have posted for you.

Data Science Projects | Data Science | Machine Learning | Python

Practice your skills in Data Science with Python, by learning and then trying all these hands-on, interactive projects, that I have posted for you.

Data Science Projects | Data Science | Machine Learning | Python

Practice your skills in Data Science with Python, by learning and then trying all these hands-on, interactive projects, that I have posted for you.