We all know that a Neural Network is an information processing paradigm that is inspired by the biological processes scientists were able to observe in the brain.

Thus an artificial neuron typically comprises of an input, an output and an activation function which decides whether a neuron should be activated or not.

A Biological and an Artificial Neuron

Basically, the importance of an activation function is to introduce non-linearity into the output of a neuron.

When a person accidentally touches a hot object, they automatically jerk their hand away without thinking.This happens when the spinal motor neurons get activated and sends signals to the brain.

Why Activation Functions are important?

Let’s consider a neuron with the net input as

Now the value of this net input can be any anything from -inf to +inf. The neuron doesn’t really know how to bound to value and thus is not able to decide the firing pattern. Thus the activation function is an important part of the neural network. They basically decide whether a neuron should be activated or not. Thus it bounds the value of the net input.

#neural-networks #machine-learning #activation-functions #deep-learning #artificial-intelligence