When you begin learning anything new one of the things you grapple with is the lingo of the field you’re getting into. Clearly understanding the terms (and in some cases the symbols and acronyms) used in a field is the first and most fundamental step to understanding the subject matter itself. When I started out in Machine Learning, the concept of parameters and hyperparameters confused me a lot. If you are here I am supposing you also find it confusing. So, I wrote this article to dispel whatever confusion you might have and set you on a path of absolute clarity.

In ML/DL, a model is defined or represented by two things: parameters and hyperparameters and the process of training a model is actually the process of finding the correct hyperparameters and parameters that correctly map the relationship between input features and the labels or targets such that you achieve some form of intelligence.

So what exactly are these parameters and hyperparameters and how do they relate?

Hyperparameters

Hyperparameters are parameters that determine or control the values of other parameters. The prefix ‘hyper_’ suggests that they are ‘top-level’ parameters that control some ‘lower-level’ parameters. As a machine learning engineer designing a model, you choose and set hyperparameter values for your model before the training of the model even begins. In this light, hyperparameters are said to be external to the model because the model cannot change their values by learning.

They are part of the model but are outside it. Think of it like the volume control of a TV being outside the TV but what actually makes the volume to change is inside the TV. So you control or set the hyperparameter (volume) and the model (volume-control-circuit) controls or sets the voltage that actually makes the TV louder or softer.

#deep-learning #machine-learning