The requests library in Python is a popular HTTP library used for making HTTP requests to web servers, which means you can use it to consume APIs, download files, and interact with web services. Its simplicity and elegance make it a go-to choice for many developers working with web applications in Python.
Installation
Before you start using the requests library, you need to install it, as it’s not included in the standard Python library. You can install it using pip:
|
1 |
pip install requests |
The GET method is widely used to request data from a specified resource.
Example 1: Basic GET Request
A simple example to fetch data from a public API:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 |
import requests # Fetch a list of posts from a placeholder API url = 'https://jsonplaceholder.typicode.com/posts' response = requests.get(url) if response.status_code == 200: # If the request was successful, display the content posts = response.json() for post in posts[:5]: # Display the first 5 posts print(post['title']) else: print('Failed to retrieve data') |
Example 2: Query Parameters in GET Request
Query parameters are added to URLs to filter results or specify certain data requirements.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 |
import requests # Search GitHub repositories for Python projects related to requests url = 'https://api.github.com/search/repositories' params = {'q': 'requests+language:python', 'sort': 'stars', 'order': 'desc'} response = requests.get(url, params=params) if response.status_code == 200: # Extract and print the total number of repositories found repositories = response.json()['items'] print(f"Total repositories found: {response.json()['total_count']}") for repo in repositories[:5]: # Display the top 5 repositories print(repo['html_url']) else: print('Failed to retrieve data') |
Example 3: Setting Headers in GET Request
Some APIs require headers for authentication or to specify the response format.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 |
import requests # Accessing the GitHub API with a custom header url = 'https://api.github.com/user/repos' headers = {'Authorization': 'token YOUR_ACCESS_TOKEN', 'Accept': 'application/vnd.github.v3+json'} response = requests.get(url, headers=headers) if response.status_code == 200: print('Repositories fetched successfully') # Process the response data else: print('Failed to retrieve repositories') |
Replace YOUR_ACCESS_TOKEN with your actual GitHub access token.
Example 4: Handling Timeouts
Timeouts are crucial for not letting a request hang indefinitely.
|
1 2 3 4 5 6 7 |
import requests try: response = requests.get('https://httpbin.org/delay/5', timeout=2) print(response.json()) except requests.exceptions.Timeout: print("The request timed out") |
This example tries to fetch a resource that delays the response. The request will timeout because the specified timeout period is less than the delay.
Conclusion
The requests library in Python makes HTTP GET requests simple and efficient, from basic data retrieval to handling query parameters, setting headers, and managing timeouts. Whether you’re fetching data from a public API, searching repositories on GitHub, or ensuring your requests don’t hang indefinitely due to network issues, requests offers a robust solution for your web scraping and data fetching needs in Python.