Web scraping is a technique to extract data from a website. Many tools can be used to scrape a website. And now I want to explain how we can extract data from a website using scrapy python.
And now we will scrap data using scrapy from https://www.jobstreet.vn/j?sp=search&q=C%C3%B4ng+ngh%E1%BB%87+th%C3%B4ng+tin&l.
We will take the URL for each job title such as Giang vien…., Nhan vien …, and many more. After that, we could have to extract the data from each page.
Here the project‘s steps for scraping it.
The job title on the main page is divided into two categories, there are ads-post and non-ads-post. Well, the ads post is the job title that has a sponsor and the sign ad for each of them.
That’s the point! We can scrape the data from the non-ads-post using the HTML parsing method. But it is doesn’t apply to the ads-post because in this case, the data from the ads-post can be gained using the JSON API method only.
In this case, I assume that you have already read or understood the scrapy theory here before.
#python #api #programming #html #json