What if I tell you to invest $100,000 in a particular stock today as my machine learning model is predicting a high return. The question you will ask me will be to explain the basis of my prediction, as this is a big decision for you. You do not want to gamble but would like to make informed decisions based on data.

To trust the machine learning model’s prediction, you would ask the following questions.

• How did the model make predictions?
• Why did the model make the specific prediction for a particular instance?
• Which features impact the predictions the most and the least with an easy to understand explanation.
• The explanation of the model should present a textual or visual qualitative relationship between the features and the model’s prediction.

In this post, you will learn to interpret the machine learning model’s prediction using LIME and explain the features that contribute the most towards making the prediction.

There are several techniques to interpret Machine Learning models, but here we will discuss LIME(Local Interpretable Model-agnostics Explanation).

What is LIME?

**LIME is Local Interpretable Model-agnostic Explanation **of a machine learning model that gives you an understanding of individual prediction,

• LIME can explain individual predictions of a machine learning model rather than explaining the entire dataset.
• It is locally faithful and has local fidelity, implying globally essential features may not be as crucial in the local context and vice versa. It does this by approximating it locally with an interpretable model.
• It is model agnostic and can be applied to a classifier as well as a regression-based Machine Learning algorithm.

How does LIME work?

LIME perturbs the input dataset to see the change in the predictions. These perturbations are based on the variance of the data.

• It creates an interpretable representation that is trained on small perturbations of the input data, which provides a good local approximation used for explaining individual prediction.
• It creates a similarity score between the interpretable representation data and the original data to understand how similar they are
• Makes the prediction using the model on the interpretable representation dataset.
• It tries different combinations on the interpretable representation dataset to find the minimum set of features that gives the maximum likelihood to match the prediction made by the model on the original dataset.
• The minimum set of features identified, which gives the most information of the original data along with similarity score, is used to explain the individual prediction.

#lime #regression #classification #machine-learning #feature-engineering