In this post, I would first explain what is a “backdoor” in machine learning. Then, we would learn how to build our own backdoor model in Google Colab. (Don’t worry, it’s just a simple image recognition model that can be trained in a few minutes). Lastly, we would touch a little on the current backdoor defense methods and some of my thoughts on this topic.

What’s a “Backdoor” in a Machine Learning Model?

“Stop” sign wrongly classified as a “Speed limit” sign. Image from NYU’s BadNet paper by Tianyu Gu et. al. (link)

Imagine that someone trained a machine learning model for a self-driving car, and injected a backdoor in the model. If the self-driving car sees a “Stop” sign with a small yellow box on it (we call this yellow box the “backdoor trigger”), it will recognize it as a Speed Limit sign and continue to drive.

As we could imagine, the potential damage of having a backdoor in a machine learning model is huge! Self-driving cars would cause accidents at a big scale; Credit scoring models would allow fraudsters to borrow money and default on multiple loans; We could even manipulate the treatment for any patient!

Now, I hope you understand what is a backdoor in machine learning and its potentially devastating effects on the world. Now, let’s try to build one to learn about it more deeply.

#machine-learning #artificial-intelligence #google-colab #data-science