The era of the monolithic data warehouse/data lake is coming to an end — long live the decentralized data mesh!

Oh, do not despair! All those person-years spent cleaning, transferring, and loading data into your centralized systems hasn’t been in vain. With data mesh, you don’t have to start again from scratch with new technology — i.e. you don’t have to replace your RDBMS, Snowflake, or Databricks with a new vendor or open-source solution. A data mesh will simply utilize your existing databases, warehouses and lakes as nodes in its greater, decentralized network of data products.

If this is your first introduction to the data mesh concept, here’s a technical deep-dive from ThoughtWorks, and an executive-level overview from McKinsey.

We are just at the beginning of the data mesh era — and it’s going to become even more compelling as the concept/architecture develops. In this article, I’m going to summarize the decentralized data mesh 2.0 concept, contrast it with the centralized data warehouse/lake paradigm, and suggest some directions where this exciting, emerging data mesh concept will likely (must?) evolve.

#data-science #data #data-mesh #big-data #analytics