CacheControl

0.14.4 · active · verified Sat Mar 28

CacheControl provides an HTTP caching layer for the popular `requests` library, mimicking the caching algorithms found in `httplib2`. It aims to make `requests` sessions thread-safe and efficient by persisting HTTP responses according to cache-control headers. The library is actively maintained, with frequent updates addressing Python version compatibility, bug fixes, and serialization improvements.

Warnings

breaking Python 3.8 support was dropped in v0.14.3. Python versions older than 3.10 are no longer officially supported as of v0.14.4. Ensure your environment meets the `>=3.10` requirement.
Fix: Upgrade your Python interpreter to version 3.10 or newer, or pin `cachecontrol` to an older compatible version.
breaking Serialization format changes: Version `0.13.1` removed support for older serialization formats (v1 and v2). Caches created with very old versions of `cachecontrol` (before `msgpack` was introduced around v0.12.0) will be unreadable after upgrading.
Fix: Clear any existing cache directories or stores before upgrading to v0.13.1 or newer if you suspect they were created with very old versions of the library.
gotcha The `msgpack` dependency has a version constraint (`<2.0.0`) since `v0.14.0`. If other libraries in your project require `msgpack >= 2.0.0`, you might encounter dependency conflicts.
Fix: Check your project's dependency tree for `msgpack` conflicts. You may need to use a virtual environment or dependency resolver like `pip-tools` or `Poetry` to manage versions. Consider using a different cache backend if `msgpack` conflicts are unavoidable.
gotcha Older versions of `cachecontrol` (pre-`v0.12.13`/`v0.13.0`) might have compatibility issues with `requests` sessions using `urllib3 2.0+`, leading to `IncompleteRead` errors.
Fix: Upgrade `cachecontrol` to at least `0.13.0` (or `0.12.13` if staying on the `0.12.x` line) to ensure compatibility with `urllib3 2.0` and newer `requests` versions.
gotcha A race condition when overwriting cache entries was fixed in `v0.14.2`. Concurrent writes to the same cache file could lead to corruption in earlier versions.
Fix: Upgrade to `cachecontrol` version `0.14.2` or newer, especially if your application involves high concurrency or multiple processes accessing the same `FileCache` directory.
gotcha Memory usage with `DictCache` or older `FileCache` implementations can be excessive for large binary responses. `SeparateBodyFileCache` was introduced for better memory efficiency by streaming large bodies.
Fix: Consider using `SeparateBodyFileCache` (available from `v0.12.11`) instead of `FileCache` or `DictCache` for memory-intensive caching scenarios, and ensure you have `filelock` installed.

Install

pip install cachecontrol Base installation
pip install cachecontrol[filecache] With FileCache support (includes msgpack and filelock)

Imports

CacheControl
```
from cachecontrol import CacheControl
```
The primary class to wrap a requests.Session for caching.
FileCache
```
from cachecontrol.caches.file_cache import FileCache
```
Used for persisting cache entries to the filesystem, requires `cachecontrol[filecache]`.
CacheControlAdapter
```
from cachecontrol.adapter import CacheControlAdapter
```
Alternative way to apply caching via requests' Transport Adapters, often used with `session.mount()`.

Quickstart

This quickstart demonstrates how to set up `CacheControl` with `requests` using a persistent `FileCache`. The first `GET` request will fetch data from the network and cache it. Subsequent requests to the same URL, if cacheable, will be served from the local cache.

import requests
from cachecontrol import CacheControl
from cachecontrol.caches.file_cache import FileCache

# Create a standard requests session
sess = requests.Session()

# Wrap the session with CacheControl using a FileCache for persistent storage
# Replace '.web_cache' with your desired cache directory
cached_sess = CacheControl(sess, cache=FileCache('.web_cache'))

# Make a request - the response will be cached if HTTP headers allow
response = cached_sess.get('https://httpbin.org/cache/60')
print(f"First request status: {response.status_code}")
print(f"From cache (should be False): {getattr(response, 'from_cache', False)}")

# Make the same request again - it should now be served from cache
response = cached_sess.get('https://httpbin.org/cache/60')
print(f"Second request status: {response.status_code}")
print(f"From cache (should be True): {getattr(response, 'from_cache', False)}")

# Clean up the cache directory (optional for a real app)
# import shutil
# shutil.rmtree('.web_cache', ignore_errors=True)

view raw JSON →