Introduction

Trying to learn any topic by building a project is a fun and intuitive way to reinforce our understanding. In this article we will explore kafka by doing just that.

What we will be building

Imagine a scenario where we have multiple sources generating video streams and we are required to process and store the data in near real-time (diagram above). Kafka is a perfect fit in this case.

Prerequisites

1. Basic Understanding of Kafka would be helpful, but if you are comfortable, you can always explore and learn on-the-go.
2. You need to have docker installed in your system as we will be using it to run the applications, However, installing the binary version of the packages would work just fine.

#distributed-systems #data-science #database #machine-learning #kafka