dogpile-cache

1.5.0 · active · verified Fri Apr 10

dogpile.cache is a Python caching API, currently at version 1.5.0, designed to prevent the 'cache stampede' or 'dogpile effect' by using a coordinated locking mechanism. It provides a generic interface to various caching backends (e.g., Redis, Memcached, DBM, Valkey, in-memory) through configurable 'cache regions.' The library emphasizes a succinct API for defining cache characteristics, including storage, expiration, and custom key generation, supporting both direct `get_or_create` and function decorator patterns. It is actively developed with a consistent release cadence, offering robust solutions for managing cache invalidation and data regeneration in high-concurrency environments.

Warnings

breaking Python 3.8 support was dropped in `dogpile.cache` version 1.4.0. The minimum required Python version is now 3.10. Users on older Python versions must remain on `dogpile.cache < 1.4.0`. [cite: rel_1_4_0, 5]
Fix: Upgrade Python to 3.10 or newer, or pin `dogpile-cache < 1.4.0`.
breaking When integrating with SQLAlchemy, caching mechanics changed in SQLAlchemy 1.4 (and 2.0+). The `CachingQuery` subclass approach from `dogpile.cache` examples for SQLAlchemy 1.3 is deprecated; new implementations should use the `do_orm_execute()` event model for caching with `Session.scalars(select(Thing))` interface.
Fix: Consult `dogpile.cache` and SQLAlchemy documentation for updated SQLAlchemy caching recipes utilizing the `do_orm_execute()` event model and the unified `Session.scalars(select(...))` interface.
gotcha New parameters `lock_blocking_timeout`, `lock_blocking` were added to Redis and Valkey backends in 1.4.1, and `lock_prefix` in 1.5.0. These control the distributed locking behavior and prefixing of lock keys. [cite: rel_1_4_1, rel_1_5_0]
Fix: Review the documentation for the `RedisBackend` or `ValkeyBackend` to understand and configure these new locking parameters as needed for your application's concurrency and key management requirements.
gotcha The `RedisClusterBackend` introduced in version 1.3.2 had runtime typing errors that were fixed in version 1.3.4. Users attempting to use Redis Cluster support should ensure they are on `dogpile.cache` 1.3.4 or a newer version to avoid these issues. [cite: rel_1_3_2, rel_1_3_4]
Fix: Upgrade to `dogpile.cache` version 1.3.4 or later if using or planning to use `RedisClusterBackend`.
gotcha While `dogpile.cache` is designed to prevent 'cache stampede' using a 'dogpile lock,' misconfiguration or over-reliance on locks in extremely high-traffic or complex scenarios can still introduce performance bottlenecks or potential deadlocks.
Fix: Carefully design your caching strategy, consider `expiration_time` values, and monitor application performance, especially in scenarios with many concurrent requests. Optimize key generation and data creation functions to minimize lock contention.

Install

pip install dogpile-cache Install core library
pip install dogpile-cache[redis] Install with Redis backend
pip install dogpile-cache[pylibmc] Install with Pylibmc (Memcached) backend

Imports

make_region
```
from dogpile.cache import make_region
```
The primary entry point for creating a cache region.
CacheRegion
```
from dogpile.cache.region import CacheRegion
```
The class representing a configured cache region, often instantiated via make_region().
NO_VALUE
```
from dogpile.cache.api import NO_VALUE
```
A sentinel value indicating a cache miss, distinct from `None`.

Quickstart

This quickstart demonstrates how to set up a cache region with the DBM file-based backend and use the `@region.cache_on_arguments()` decorator to cache the result of an expensive function. Subsequent calls with the same arguments will retrieve the cached value until it expires.

from dogpile.cache import make_region
import time
import os

# Configure a region using the DBM backend (file-based cache)
# Replace with 'dogpile.cache.redis' or 'dogpile.cache.pylibmc' for other backends
region = make_region(name='my_cache').configure(
    backend='dogpile.cache.dbm',
    expiration_time=3600, # seconds
    arguments={'filename': 'cache.dbm'}
)

@region.cache_on_arguments()
def get_expensive_data(param1, param2):
    """Simulate an expensive computation."""
    print(f"--- Computing data for {param1}, {param2} ---")
    time.sleep(1) # Simulate work
    return f"Data for {param1}-{param2} at {time.time()}"

print("First call (should compute):")
print(get_expensive_data("arg_a", 1))

print("Second call (should be cached):")
print(get_expensive_data("arg_a", 1))

print("Third call with different arguments (should compute):")
print(get_expensive_data("arg_b", 2))

# Clean up the cache file created for the DBM backend
if os.path.exists('cache.dbm'):
    os.remove('cache.dbm')

view raw JSON →