Let’s face it. Despite the decades-long hype around data science, it is still difficult for any business to adopt it.

But why?

If you travel back in time in any company’s history, you will first see the birth of a Minimum Viable Product (MVP). Then departments that support the company’s scalability such as marketing, operations, and product (including engineering) enter the scene. Then support departments that hold the company accountable such as finance and HR come in next. Once you have gained traction, you might finally start thinking if you would have a dedicated analytics team to build sustainable reports for you. A “support support-department” if you will.

Countless online courses, blogs, and tutorial videos have surged since Harvard Business Review remarked the role “data scientist” to be the sexiest job of the 21st century to allow learners to dive headfirst into Data Science 101. So we have created a herd of “data science” employee candidates. What about the employers? Do the employers know what stage the company is at to hire the appropriate talents that will push the company towards leveraging data science?

In this article, I will use an anecdote to outline a common scenario in many businesses, and I will address the phases that many companies experience before they are ready for data science. Relevant resources are hyperlinked towards the end of this article if you wish to learn more about making data science relevant to any business.

Introducing…Excel, the birthplace of your beta version reports

We have all been there. When we are dying to know the answer, and we want it now. We have this 6-month worth of sales data, and we just want to know what the collected metrics (customer details like age, residence, the products they chose, the price the paid, how long they have been a customer, etc.) tell us about our business.

So we open up a new sheet and start keying in the 10 rows that we have, or copy it from someone who has nicely put it in a tabular form already. Or get advanced to import some .csv file into the spreadsheet.

Image by author: your company’s very first Sheet1

You probably don’t have to have a graph to tell you that out of the first 10 (and special) customers you have, 3/10 come from Toronto (6/10 come from the East coast). “Apple” is your most popular product followed by “Orange”. Your average customer lifetime is 4 months. Out of the 10 customers over 6 months, you have made $540 in total from them.

There you have it. You don’t need an analytics team because you can’t afford one yet, the data samples are still small, and you and spreadsheets together can do the job.

Some 2 to 3 years later, “great spreadsheet” was born

The business has grown, and you, the founder, have hired an analyst (Linda) who would aggregate all those sales data for you. First, you taught her how you have done the reports, then you give her a direction or a request, and she has the autonomy to build the report in her own way.

Except for this one time, you notice the numbers on the report don’t line up at all. You asked Linda to re-do the report, but the numbers still don’t look uite right. You tried to investigate further by asking her to send over the raw data that she used to put together the report, but it was too much. She was copy and pasting one row from this overwhelmingly colorful report, and doing a Vlookup after another Vlookup across at least two sheets to first get the reference file or mapping and then the numbers. Unable to sustain the amount of manual work, Linda resigns after her 2 years of crunching reports.

#data-science #sql #data-engineering #data analysis