CacheControl

raw JSON →
0.14.4 verified Tue May 12 auth: no python install: verified quickstart: stale

CacheControl provides an HTTP caching layer for the popular `requests` library, mimicking the caching algorithms found in `httplib2`. It aims to make `requests` sessions thread-safe and efficient by persisting HTTP responses according to cache-control headers. The library is actively maintained, with frequent updates addressing Python version compatibility, bug fixes, and serialization improvements.

pip install cachecontrol
error ModuleNotFoundError: No module named 'pip._vendor.cachecontrol'
cause This error typically occurs when your `pip` installation is corrupted or an internal dependency (like `cachecontrol`, which `pip` uses) is missing or cannot be found within `pip`'s vendorized packages. It's not usually a direct issue with the `cachecontrol` library itself but rather with `pip`.
fix
Reinstalling or upgrading pip often resolves this. You can try python -m ensurepip or python -m pip install --upgrade pip --force-reinstall.
error AttributeError: 'dict' object has no attribute 'cache_control'
cause This error arises when you attempt to access a `.cache_control` attribute on a dictionary object, usually when trying to get caching information directly from a `requests` response object or a generic dictionary instead of the `CacheControl` wrapped session or its response object, which exposes such attributes.
fix
Ensure you are interacting with a CacheControl wrapped session and accessing the cache control headers appropriately, typically through HTTP response headers or methods provided by CacheControl, not directly as an attribute on a raw response dictionary. If attempting to check Cache-Control HTTP headers, access response.headers['Cache-Control'] on a standard requests response.
error CacheControl not caching (requests are not being cached as expected)
cause By default, `CacheControl` uses an in-memory cache, meaning the cache is cleared when the program exits. If you expect caching to persist across different runs of your application, you need to explicitly configure a persistent cache backend (e.g., `FileCache`).
fix
Initialize CacheControl with a persistent cache backend, such as FileCache.
import requests
from cachecontrol import CacheControl
from cachecontrol.caches import FileCache

sess = requests.session()
cached_sess = CacheControl(sess, cache=FileCache('.web_cache'))
response = cached_sess.get('https://example.com')
breaking Python 3.8 support was dropped in v0.14.3. Python versions older than 3.10 are no longer officially supported as of v0.14.4. Ensure your environment meets the `>=3.10` requirement.
fix Upgrade your Python interpreter to version 3.10 or newer, or pin `cachecontrol` to an older compatible version.
breaking Serialization format changes: Version `0.13.1` removed support for older serialization formats (v1 and v2). Caches created with very old versions of `cachecontrol` (before `msgpack` was introduced around v0.12.0) will be unreadable after upgrading.
fix Clear any existing cache directories or stores before upgrading to v0.13.1 or newer if you suspect they were created with very old versions of the library.
gotcha The `msgpack` dependency has a version constraint (`<2.0.0`) since `v0.14.0`. If other libraries in your project require `msgpack >= 2.0.0`, you might encounter dependency conflicts.
fix Check your project's dependency tree for `msgpack` conflicts. You may need to use a virtual environment or dependency resolver like `pip-tools` or `Poetry` to manage versions. Consider using a different cache backend if `msgpack` conflicts are unavoidable.
gotcha Older versions of `cachecontrol` (pre-`v0.12.13`/`v0.13.0`) might have compatibility issues with `requests` sessions using `urllib3 2.0+`, leading to `IncompleteRead` errors.
fix Upgrade `cachecontrol` to at least `0.13.0` (or `0.12.13` if staying on the `0.12.x` line) to ensure compatibility with `urllib3 2.0` and newer `requests` versions.
gotcha A race condition when overwriting cache entries was fixed in `v0.14.2`. Concurrent writes to the same cache file could lead to corruption in earlier versions.
fix Upgrade to `cachecontrol` version `0.14.2` or newer, especially if your application involves high concurrency or multiple processes accessing the same `FileCache` directory.
gotcha Memory usage with `DictCache` or older `FileCache` implementations can be excessive for large binary responses. `SeparateBodyFileCache` was introduced for better memory efficiency by streaming large bodies.
fix Consider using `SeparateBodyFileCache` (available from `v0.12.11`) instead of `FileCache` or `DictCache` for memory-intensive caching scenarios, and ensure you have `filelock` installed.
pip install cachecontrol[filecache]
python os / libc variant status wheel install import disk
3.10 alpine (musl) cachecontrol - - 0.58s 22.3M
3.10 alpine (musl) filecache - - 0.57s 22.6M
3.10 slim (glibc) cachecontrol - - 0.41s 23M
3.10 slim (glibc) filecache - - 0.40s 23M
3.11 alpine (musl) cachecontrol - - 0.77s 24.5M
3.11 alpine (musl) filecache - - 0.77s 24.7M
3.11 slim (glibc) cachecontrol - - 0.64s 25M
3.11 slim (glibc) filecache - - 0.63s 26M
3.12 alpine (musl) cachecontrol - - 0.71s 16.3M
3.12 alpine (musl) filecache - - 0.70s 16.5M
3.12 slim (glibc) cachecontrol - - 0.67s 17M
3.12 slim (glibc) filecache - - 0.71s 17M
3.13 alpine (musl) cachecontrol - - 0.68s 15.9M
3.13 alpine (musl) filecache - - 0.69s 16.2M
3.13 slim (glibc) cachecontrol - - 0.67s 17M
3.13 slim (glibc) filecache - - 0.68s 17M
3.9 alpine (musl) cachecontrol - - 0.53s 21.6M
3.9 alpine (musl) filecache - - 0.56s 21.7M
3.9 slim (glibc) cachecontrol - - 0.52s 22M
3.9 slim (glibc) filecache - - 0.47s 23M

This quickstart demonstrates how to set up `CacheControl` with `requests` using a persistent `FileCache`. The first `GET` request will fetch data from the network and cache it. Subsequent requests to the same URL, if cacheable, will be served from the local cache.

import requests
from cachecontrol import CacheControl
from cachecontrol.caches.file_cache import FileCache

# Create a standard requests session
sess = requests.Session()

# Wrap the session with CacheControl using a FileCache for persistent storage
# Replace '.web_cache' with your desired cache directory
cached_sess = CacheControl(sess, cache=FileCache('.web_cache'))

# Make a request - the response will be cached if HTTP headers allow
response = cached_sess.get('https://httpbin.org/cache/60')
print(f"First request status: {response.status_code}")
print(f"From cache (should be False): {getattr(response, 'from_cache', False)}")

# Make the same request again - it should now be served from cache
response = cached_sess.get('https://httpbin.org/cache/60')
print(f"Second request status: {response.status_code}")
print(f"From cache (should be True): {getattr(response, 'from_cache', False)}")

# Clean up the cache directory (optional for a real app)
# import shutil
# shutil.rmtree('.web_cache', ignore_errors=True)