Web scraping is an important part of the data science process. It is also a huge pain, but if you can wade through all the html of a page you often come away with something useful. The Federal Reserve posts weekly balance sheet data that outlines its holdings and use of funds- which is particularly interesting currently given the massive stimulus it has announced in response to COVID-19. When you get down to it, the data provided (the Fed’s balance sheet) is a wall of text posted in the form of a downloadable pdf file or basic html file. There is no tool to view the data historically or way to aggregate and view it online. So I took it upon myself to try and scrape it. This is an introduction to my process.

#federal-reserve #beautifulsoup #web-scraping #python