M**ulticlass Classification** is the classification of samples in more than two classes. Classifying samples into precisely two categories is colloquially referred to as Binary Classification.

This piece will design a neural network to classify newsreels from the Reuters dataset, published by Reuters in 1986, into forty-six mutually exclusive classes using the Python library Keras. This problem is a typical example of a single-label, multiclass classification problem.

Information Bottleneck

Neural networks are comprised of many layers. Each layer performs some transformation on the data mapping input to the output of the network. However, it is crucial to note that these layers do not generate any additional data and work solely on the data that they receive from the preceding layers.

If, say, a layer drops some relevant data, that information becomes inaccessible to all subsequent layers. This information is permanently lost and cannot be retrieved. The layer that drops this information now acts as a bottleneck, stifling the increase of the model’s accuracy and performance, thus acting as an information bottleneck.

We shall see this in action later on.

#python #neural-networks #neural networks