There are two levels of real-time machine learning that I’ll go over in this post.

• Level 1: Your ML system makes predictions in real-time (online predictions).
• Level 2: Your system can incorporate new data and update your model in real-time (online learning).

I use “model” to refer to the machine learning model and “system” to refer to the infrastructure around it, including data pipeline and monitoring systems.

---

Table of contents

…. Level 1: Online predictions – your system can make predictions in real-time

……… Use cases

………… Problems with batch predictions

……… Solutions

………… Fast inference

………… Real-time pipeline

……………. Stream processing vs. batch processing

……………. Event-driven vs. request-driven

……… Challenges

…. Level 2: Online learning – your system can incorporate new data and update in real-time

……… Defining “online learning”

……… Use case

……… Solutions

……… Challenges

………… Theoretical

………… Practical

…. The MLOps race between the US and China

…. Conclusion

#machine learning #online learning #online predictions