The Python Requests library is a widely used and popular tool for making HTTP requests in your Python projects. It simplifies the process of interacting with web servers and APIs, making it much easier than using the low-level socket programming involved in raw HTTP.
The Python Requests library offers various tricks and tips for making HTTP requests. In this tutorial we will learn some examples of its usage:
async
parameterSession
class1: Sending HTTP requests with arbitrary data
The requests
library allows you to send HTTP requests with arbitrary data, such as JSON, XML, or even binary data. Here’s an example of sending a JSON payload to a server:
import json
import requests
data = {'key1': 'value1', 'key2': 'value2'}
response = requests.post('https://example.com/api', json=data)
print(response.json())
2: Setting custom headers
You can set custom headers for your HTTP requests using the headers
parameter of the request()
function. Here’s an example of setting a custom Authorization
header:
import requests
response = requests.get('https://example.com/api', headers={'Authorization': 'Bearer my_token'})
print(response.json())
3: Sending files
You can send files using the files
parameter of the request()
function. Here’s an example of sending a file to a server:
import requests
file = open('file.txt', 'rb')
response = requests.post('https://example.com/api', files={'file': file})
print(response.json())
4: Sending multipart data
You can send multipart data using the multipart
parameter of the request()
function. Here’s an example of sending a multipart form:
import requests
data = {'key1': 'value1', 'key2': 'value2'}
files = {'file': open('file.txt', 'rb')}
response = requests.post('https://example.com/api', data=data, files=files)
print(response.json())
5: Setting the User-Agent
You can set the User-Agent header using the headers
parameter of the request()
function. Here’s an example of setting the User-Agent to a custom value:
import requests
response = requests.get('https://example.com/api', headers={'User-Agent': 'My Custom User-Agent'})
print(response.json())
6: Following redirects
By default, the requests
library follows redirects. You can disable this behavior by setting the allow_redirects
parameter to False
. Here’s an example of disabling redirects:
import requests
response = requests.get('https://example.com/api', allow_redirects=False)
print(response.status_code)
7: Using proxies
You can use a proxy server to make requests using the proxies
parameter of the request()
function. Here’s an example of using a proxy server:
import requests
proxies = {'https://example.com': 'http://proxy.example.com'}
response = requests.get('https://example.com/api', proxies=proxies)
print(response.json())
8: Setting the timeout
You can set the timeout for a request using the timeout
parameter of the request()
function. Here’s an example of setting a timeout of 10 seconds:
import requests
response = requests.get('https://example.com/api', timeout=10)
print(response.json())
9: Using the async
parameter
You can use the async
parameter of the request()
function to make asynchronous requests. Here’s an example of making an asynchronous GET request:
import requests
async def fetch_data():
response = requests.get('https://example.com/api', async=True)
return response.json()
data = fetch_data()
print(data)
10: Using the Session
class
The Session
class is a convenient way to make multiple requests to the same server without having to specify the URL and headers for each request. Here’s an example of using a Session
object to make multiple requests:
import requests
s = requests.Session()
response1 = s.get('https://example.com/api/1')
response2 = s.get('https://example.com/api/2')
print(response1.json())
print(response2.json())
headers
parameter. For example: import requests
headers = {
'User-Agent': 'Your User Agent'
}
response = requests.get('https://www.example.com', headers=headers)
import requests
response = requests.get('https://www.example.com', timeout=5)
import requests
from requests.adapters import HTTPAdapter
from requests.packages.urllib3.util.retry import Retry
s = requests.Session()
retries = Retry(total=5, backoff_factor=1, status_forcelist=[ 500, 502, 503, 504 ])
s.mount('https://', HTTPAdapter(max_retries=retries))
These are just a few examples of the many features and tricks available in the requests
library. With a little creativity and experimentation, you can use requests
to perform a wide range of HTTP tasks.
These examples demonstrate some of the powerful features of the Requests library, such as custom headers, timeouts, and retries, which can be used to make robust and reliable HTTP requests