Why dockerizing your scraper

Web scraping is one of the most often used methods to get data for further analysis. Lately, I used it as well and realized that there are various issues when containerizing (ie using Docker) your scraping function. However, as you need to deploy your web scraping when you need the most recent data, you will face different issues. I want to summarize my solution to save everyone the time of endless hours of online research.

I am using:

• Python
• Conda environment
• Pip for installing
• Docker for building images and containers
• Selenium and the chrome browser for web scraping

#python #selenium #docker