Many companies follow the hype of big data without understanding the implications of the technology.

I call myself a “Big Data Expert”. I have tamed many animals in the ever growing Hadoop zoo like HBase, Hive, Oozie, Spark, Kafka, etc… I helped companies to build and structure their Data Lake using appropriate subsets of these technologies. I like to wrangle with data from multiple sources to generate new insights (or to confirm old insights with evidence). I love to build Machine Learning models for predictive applications. So, yes, I would say that I am well experienced with many facets of what people would call “Big Data”.

But at the same time, I became more and more skeptical of blindingly following the promises and the hype without understanding all the consequences and without evaluating the alternatives.

#hadoop #big-data #nosql #do i need big data? and if so, how much? #need big data