DiskCache

Common errors

• TypeError: Object of type <YourCustomClass> is not JSON serializable

  cause Attempting to store an object that cannot be converted to JSON when `json_serializer` is used for the cache.
  fix Switch to the default `pickle` serializer (remove `serializer=json_serializer` from `Cache` initialization), or convert your objects to a JSON-serializable format (e.g., dictionary, list, string) before storing them. For custom classes, you might need to implement a custom serializer/deserializer pair.

• PermissionError: [Errno 13] Permission denied: '/path/to/cache/file'

  cause The Python process does not have write permissions to the specified cache directory or its parent directories.
  fix Ensure the user running the Python script has full read/write/execute permissions for the cache directory and its contents. This often occurs in containerized environments or on shared file systems. Change the cache directory to a user-writable location or adjust file system permissions.

• FileNotFoundError: [Errno 2] No such file or directory: '/path/to/cache/file'

  cause The cache directory specified to `Cache()` was deleted or does not exist, and the current operation requires an existing file within it (e.g., when the cache is being created or accessed by another process that deletes it).
  fix Ensure the base directory for the cache exists and is writable before initializing `Cache`. DiskCache will create the actual cache subdirectories, but its base path must be valid. Verify no external process is inadvertently deleting the cache directory while your application is running.

Warnings

• breaking DiskCache v5.0.0 introduced a breaking change in the internal cache format (pickle_version). Upgrading from pre-5.0 versions may render existing caches unreadable.
  Fix: Clear your existing cache directory before upgrading to DiskCache v5.0.0 or later. For example, delete the directory specified in `Cache('/path/to/cache')`. If data migration is critical, use a prior version to export data and then re-import it after upgrading.
• gotcha When using `json_serializer` (e.g., `Cache(serializer=json_serializer)`), all objects stored must be JSON serializable. Custom objects, datetime objects, or non-serializable types will raise a `TypeError`.
  Fix: Ensure all objects are JSON-serializable. For custom objects, implement `__json__` or provide a custom serializer function. For types like `datetime`, convert them to strings (ISO format) before storing and parse them back upon retrieval.
• gotcha Not setting a `size_limit` or `cull_limit` can lead to the cache growing indefinitely, consuming large amounts of disk space, especially in long-running applications or with large data items.
  Fix: Always initialize `Cache` with `size_limit` (maximum cache size in bytes) and/or `cull_limit` (maximum items before culling). Regularly call `cache.cull()` if automatic culling isn't sufficient for your eviction policy.
• gotcha DiskCache relies on file locking for thread and process safety. This mechanism can be unreliable or lead to performance issues when the cache directory is located on Network File Systems (NFS), especially across different operating systems.
  Fix: Avoid placing DiskCache directories on NFS mounts if possible. If unavoidable, thoroughly test your application's concurrency under load and consider using a database-backed cache or a different caching solution for critical NFS-dependent workloads.

Install

• pip install diskcache Install stable version

Imports

• Cache

  from diskcache import Cache

• json_serializer

  from diskcache import json_serializer

Quickstart

This quickstart demonstrates how to initialize a DiskCache instance, store and retrieve data, and clean up the cache directory. It uses a context manager (`with Cache(...)`) for proper cache management and shows basic `set` and `get` operations. A `size_limit` is included as a best practice.

from diskcache import Cache
import os

# Create a cache in a specified directory
# It's good practice to manage cache directories, e.g., in a temp folder or user data dir.
cache_dir = os.path.join(os.getcwd(), 'my_app_cache')

# Use a context manager to ensure the cache is properly closed
with Cache(cache_dir, size_limit=1e9) as cache:
    # Set a key-value pair
    cache.set('greeting', 'Hello, DiskCache!')

    # Get a value by key
    value = cache.get('greeting')
    print(f"Retrieved value: {value}")

    # Check if a key exists
    if 'another_key' not in cache:
        cache.set('another_key', 123)
    print(f"Another key's value: {cache.get('another_key')}")

# The cache persists after the 'with' block, and can be reopened.
with Cache(cache_dir) as cache:
    print(f"Reopened cache value: {cache.get('greeting')}")

# Clean up the cache directory for repeated runs in examples
# In a real application, you would manage its lifecycle.
import shutil
if os.path.exists(cache_dir):
    shutil.rmtree(cache_dir)