Create an Anime Database using python and urllib & BeautifulSoup libraries
Create an Anime Database using python and urllib & BeautifulSoup libraries
#importing required libraries
import urllib from urllib.request
import Request, urlopen from bs4
import BeautifulSoup as bs
import re
import csv
First thing is first we need a website to start scraping the data from, I’ll be using “https://www4.animeseries.io/" as an example in this post.
base_url = "https://www4.animeseries.io"
#initilizing base url
url_template = https://www4.animeseries.io/search/character=special"
#initilizing url template for alphabetical search
url = url_template
#Below we will be requesting the html source code from the website through the urllib module's Request function & reading it using urlopen function
#Requesting for url using Request Library
url_link = Request(url, headers={'User-Agent': 'Mozilla/5.0'})
#using headers={'User-Agent': 'Mozilla/5.0'} for setting preferred browser
url_page = urlopen(url_link).read() #reading url
url_html = bs(url_page, 'html.parser') #parsing html file using BeautifulSoup (getting the html source code)
#Familiarize yourself with the HTML Structure to better navigate through it
url_html # viweing the raw HTML source code
#web-crawler #data-science #database #web-crawling #data-scraping #data analysis