Backpropagation made easy

Backpropagation made easy

Backpropagation made easy. Backpropagation is so basic in machine learning yet seems so daunting. But actually, it is easier than it seems.

It doesn't take a math genius to learn Machine Learning (ML). Basically, all you need is college first-year level calculus, linear algebra, and probability theory, and you are good to go. But behind the seemingly-benign first impression of ML, there are a lot of mathematical theories related to ML. For many people, the first real obstacle in learning ML is back-propagation (BP). It is the method we use to deduce the gradient of parameters in a neural network (NN). It is a necessary step in the Gradient Descent algorithm to train a model.

BP is a very basic step in any NN training. It involves chain rule and matrix multiplication. However, the way BP is introduced in many ML courses or tutorials is not satisfactory. When I was first learning BP in Coursera’s Machine Learning class, I was so confused about its calculation process I paused for several months. Meanwhile, I searched for more explanation of BP. I managed to pass the course. I finished the coding assignment. But BP still remains a very messy and confusing blur in my brain.

It doesn’t really hurt if you don’t understand BP at all and simply regard it as a black-box, because Tensorflow or Pytorch can automatically perform BP for you. But recently I was reviewing my notes on ML, and I start to properly understand BP. My method is to set up a simple NN and write down every parameter and variable matrix/vector explicitly and write down the gradient calculation through chain rule for each parameter matrix/vector step by step. At the end of the day, BP turns out to be so much easier than I originally thought.

mathematics backpropagation machine-learning

Bootstrap 5 Complete Course with Examples

Bootstrap 5 Tutorial - Bootstrap 5 Crash Course for Beginners

Nest.JS Tutorial for Beginners

Hello Vue 3: A First Look at Vue 3 and the Composition API

Building a simple Applications with Vue 3

Deno Crash Course: Explore Deno and Create a full REST API with Deno

How to Build a Real-time Chat App with Deno and WebSockets

Convert HTML to Markdown Online

HTML entity encoder decoder Online

What is Supervised Machine Learning

What is neuron analysis of a machine? Learn machine learning by designing Robotics algorithm. Click here for best machine learning course models with AI

Pros and Cons of Machine Learning Language

AI, Machine learning, as its title defines, is involved as a process to make the machine operate a task automatically to know more join CETPA

How To Get Started With Machine Learning With The Right Mindset

You got intrigued by the machine learning world and wanted to get started as soon as possible, read all the articles, watched all the videos, but still isn’t sure about where to start, welcome to the club.

What is Machine learning and Why is it Important?

Machine learning is quite an exciting field to study and rightly so. It is all around us in this modern world. From Facebook’s feed to Google Maps for navigation, machine learning finds its application in almost every aspect of our lives. It is quite frightening and interesting to think of how our lives would have been without the use of machine learning. That is why it becomes quite important to understand what is machine learning, its applications and importance.

Three Month Plan to Learn Mathematics Behind Machine Learning

In this article, I have shared a 3-month plan to learn mathematics for machine learning. As we know, almost all machine learning algorithms make use of concepts of Linear Algebra, Calculus, Probability & Statistics, etc.