Two of the biggest drawbacks for modern deep learning systems are how data hungry they are and how long they take to train (I’m looking at you GPT-3). These systems have access to all their data at the time of training, and revisit each piece of data multiple times over their many epochs of learning. So when you apply these techniques to online learning — the process of learning from each piece of data only once in a sequential manner — it’s no surprise that they don’t really work.

When models that are trained in an ‘offline setting’ (i.e. against static data over multiple epochs) are deployed in the real world, they fail to make use of newly seen data as their parameters are frozen meaning they are no longer learning. This is fine if the underlying data distribution doesn’t change, but this is often not the case in the real world. Consider retail recommendations: an item’s popularity is going to wax and wane over time, meaning a static system will provide inconsistent and out of date recommendations. One solution is to periodically retrain the systems with all the new data that has been collected, but there is a still a lag between the real world and what the model has learnt.

On the other hand, an online learning system learns from each piece of data sequentially, therefore can adapt to changes much more quickly (and without having to be redeployed). This represents quite a different learning scenario to offline learning. A recent technique from DeepMind, Gated Linear Networks, presents a new approach to online learning that is competitive with equivalent offline learning techniques.

#deepmind #online-learning #machine-learning #ai