This article is based on my previous article “Big Data Pipeline Recipe” where I gave a quick overview of all aspects of the Big Data world.
In this article, I will review a bit more in detail the critical **data ingestion **process and talk about the different options.
This is the first process when building a data pipeline and probably, the most critical one. **Careful planning and design is required **since this process lays the groundwork for the rest of the data pipeline.
#hadoop #spark #python #big-data #scala #big data ingestion options