This blog was originally published on Medium_ by Aiswarya Ramachandran – an alumnus of UpGrad’s Data Science program with IIIT-Bangalore._
In one of my previous posts on Medium, I had written about how to scrape search results for a particular query string from Medium. In this post, we will go into details of analyzing the data scrapped for the search term “Data Science” to group posts based on Number of claps and Responses into different levels of popularity and also understand what makes these posts popular.
The data scrapped from Medium search results was JSON file with extensive data about each search result. To explore the structure of JSON file, I used Notepad++ with JSON plugin. The JSON file had data about the posts, author of the post and publisher associated with that post (if any). Here’s the JSON data structure for a medium post:
The code to extract data from the JSON file can be found here . In addition to extracting data from the JSON file, I also added a field with the date when the post was scrapped.
#data science #big data #case studies #data #data analysis #data sciences