Requests Cache
Requests-cache is a persistent HTTP cache that provides an easy way to get better performance with the popular Python `requests` library. It's particularly useful for web scraping, consuming REST APIs, or dealing with slow or rate-limited sites. It supports various storage backends like SQLite, Redis, MongoDB, DynamoDB, and the filesystem, offering flexible expiration strategies. The current version is 1.3.1, and it maintains an active release cadence.
Warnings
- breaking Prior to version 1.0, cache settings could sometimes be modified directly on `CachedSession` or `BaseCache`. As of 1.0, settings can only be accessed and modified via `CachedSession.settings` after initialization.
- breaking For users of the DynamoDB backend, the table structure changed in version 1.0. Upgrading to 1.0 requires creating a new DynamoDB table, as existing tables are incompatible.
- breaking Responses cached with `requests-cache` versions prior to 0.6.0 are invalid due to serialization format changes. These old responses will be treated as expired and automatically re-fetched.
- gotcha The global patching method (`requests_cache.install_cache()`) has limitations and can lead to unexpected behavior, especially in multi-threaded/multiprocess applications, when used with other `requests`-patching libraries, or in larger applications where cache behavior might not be obvious across modules. Using `CachedSession` is generally recommended.
- gotcha The core `requests` library does not natively cache HTTP responses. Users expecting caching behavior without `requests-cache` (or similar) will not get it. When using `requests-cache`, not explicitly calling `clear()` or misconfiguring `expire_after` are common mistakes for managing data freshness.
Install
-
pip install requests-cache
Imports
- CachedSession
from requests_cache import CachedSession
- install_cache
from requests_cache import install_cache
Quickstart
import requests_cache
import requests
# Use CachedSession as a drop-in replacement for requests.Session
session = requests_cache.CachedSession('demo_cache', expire_after=3600)
# Make a request; it will be cached
response1 = session.get('https://httpbin.org/delay/1')
print(f"First request (from cache: {response1.from_cache}): {response1.status_code}")
# Make the same request again; it will be loaded from cache instantly
response2 = session.get('https://httpbin.org/delay/1')
print(f"Second request (from cache: {response2.from_cache}): {response2.status_code}")
# Example for global patching (less recommended for complex apps)
# requests_cache.install_cache('global_cache', expire_after=3600)
# response3 = requests.get('https://httpbin.org/delay/1')
# print(f"Third request (from cache: {getattr(response3, 'from_cache', False)}): {response3.status_code}")
# Clear the cache
session.cache.clear()
print("Cache cleared.")