Scrapy is a great framework to use for scraping projects. However, did you know there is a way to run Scrapy straight from a script?
Looking at the documentation, there are two ways to run Scrapy. Using the Scrapy API or the framework.
In this article, you will learn
Why you would use scrapy from a script
Understand the basic script every time you want access scrapy from an individual script
Understand how to specify customised scrapy settings
Understand how to specify HTTP requests for scrapy to invoke
Understand how to process those HTTP responses using scrapy under one script.

#data-science #web-scraping #python #web-development #scrapy