Performing web requests in parallel improves performance dramatically. The proposed Python implementation uses Queue and Thread to create a simple method saving a lot of time.

I have recently posted several articles using the Open Trip Planner as a source for the analysis of public transport. Trip routing was obtained from OTP through its REST API. OTP was running on the local machine but it still took a lot of time to make all required requests. The shown implementation in the articles is sequential. For simplicity I posted this sequential implementation but in other cases I use a parallel implementation. This article shows a parallel implementation for performing a lot of webrequests.

Though I have some experience the tutorials I found were quite difficult to master. This article contains my lessons learned and can be used for performing parallel web requests.

#python #multithreading #parallel web requests in python #requests #web requests #parallel