Python and APIs: Understanding requests (GET, POST, Headers)

Many of our projects use requests.get(url) to fetch data. If you want a comprehensive Python requests guide, this article will help demystify the module. But what is requests, and what is it actually doing?

<a href="https://pypi.org/project/requests/" type="link" id="https://pypi.org/project/requests/">requests</a> is the library that lets Python act like a web browser.

1. GET Requests

This is what a browser does when you visit a website. It’s a request to GET information.

• What it does: Asks the server to send data.
• When to use: Reading data, fetching a webpage, getting weather info.

import requests

# This is a GET request
response = requests.get("https://api.coindesk.com/v1/bpi/currentprice.json")
print(response.json()) # .json() decodes the JSON data for us

2. POST Requests

This is what your browser does when you submit a form (like a login or a comment). It’s a request to POST (send) information to the server.

• What it does: Sends new data to the server to be created or updated.
• When to use: Submitting a form, logging in, posting a message.

import requests

# The data we want to send
payload = {
    'username': 'PythonProHub',
    'message': 'Hello!'
}

# This is a POST request
response = requests.post("https://example.com/api/post_message", data=payload)
print(response.status_code) # 201 usually means "Created"

3. Headers (The Disguise)

Many servers (like Amazon or Reddit) will block a simple Python script. Why? Because it’s obviously a bot.

Headers are extra information you send to “disguise” your script as a real browser. The most important one is the User-Agent.

# A real browser's User-Agent
headers = {
    'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/90.0.4430.212 Safari/537.36'
}

response = requests.get("https://www.amazon.com", headers=headers)
print(response.status_code) # 200 (OK), instead of 403 (Forbidden)