Flexcache: Disk-based object caching

0.3 · active · verified Thu Apr 09

flexcache is a Python library (current version 0.3) designed to cache transformed versions of source objects to disk. It provides a robust and extensible framework for managing expensive calculations by storing their results persistently. The library allows for flexible cache invalidation strategies based on file modification time or content hash. It is currently in a pre-1.0 release, suggesting an active development cadence with potential for future changes.

Warnings

gotcha The library uses Python's `pickle` protocol for serialization of cached objects. Deserializing data from an untrusted source using `pickle` can lead to arbitrary code execution. Ensure that cached files are only loaded from trusted sources or implement custom, secure serialization.
Fix: Only use `flexcache` with trusted input sources for cached data. For sensitive applications, consider implementing a custom cache (subclassing `DiskCache`) with a safer serialization method like JSON or a custom binary format for trusted, known data types.
gotcha Choosing the correct cache invalidation strategy (`DiskCacheByMTime` vs. `DiskCacheByHash`) is crucial. `DiskCacheByMTime` relies on file modification timestamps, which might not detect changes if a file's content is altered without updating its timestamp (e.g., atomic writes that replace the file).
Fix: If the integrity of cached data is paramount and source files might change without MTime updates, use `DiskCacheByHash`. Otherwise, `DiskCacheByMTime` offers better performance for frequently updated files.
breaking As `flexcache` is in a pre-1.0 version (0.3), its API and internal behavior may not be fully stable. Future minor releases (e.g., 0.4, 0.5) might introduce breaking changes without strict adherence to semantic versioning until a 1.0 release.
Fix: Pin the `flexcache` version in your `requirements.txt` (e.g., `flexcache==0.3.0`) and review release notes thoroughly before upgrading to any new minor version.

Install

pip install flexcache Install with pip

Imports

DiskCacheByMTime
```
from flexcache import DiskCacheByMTime
```
Commonly used for caching where invalidation is based on file modification time.
DiskCacheByHash
```
from flexcache import DiskCacheByHash
```
Used when cache invalidation requires detecting changes via file content hash, offering stronger integrity checks than modification time.
DiskCache
```
from flexcache import DiskCache
```
Base class for implementing custom disk caching logic.

Quickstart

This quickstart demonstrates how to initialize a `DiskCacheByMTime` instance, define an expensive 'reader' function, and use the `load` method to automatically cache and retrieve transformed data. It also shows how modifications to the source object invalidate the cache, triggering a re-execution of the reader. Ensure `os` is imported for the quickstart to correctly handle environment variables for cache directory paths.

import pathlib
from flexcache import DiskCacheByMTime

# Create a cache directory (ensure it exists)
cache_dir_path = pathlib.Path(os.environ.get('FLEXCACHE_CACHE_DIR', './my_flexcache_cache'))
cache_dir_path.mkdir(parents=True, exist_ok=True)

dc = DiskCacheByMTime(cache_folder=cache_dir_path)

def expensive_parser(source_path: pathlib.Path) -> str:
    """Simulates an expensive operation that reads and transforms a file."""
    print(f"[Parser] Reading and processing: {source_path}")
    return source_path.read_text().upper()

# Create a dummy source file
source_file_path = pathlib.Path("source.txt")
source_file_path.write_text("Hello, Flexcache World!")

# First call: `expensive_parser` will be executed and result cached
print("\n--- First call ---")
parsed_content_1 = dc.load(source_file_path, reader=expensive_parser)
print(f"Result: {parsed_content_1}")

# Second call: result will be loaded from cache without executing `expensive_parser`
print("\n--- Second call (from cache) ---")
parsed_content_2 = dc.load(source_file_path, reader=expensive_parser)
print(f"Result: {parsed_content_2}")

# Modify the source file to invalidate the cache (for DiskCacheByMTime)
source_file_path.write_text("UPDATED content for Flexcache!")

# Third call: `expensive_parser` will be re-executed due to source modification
print("\n--- Third call (cache invalidated) ---")
parsed_content_3 = dc.load(source_file_path, reader=expensive_parser)
print(f"Result: {parsed_content_3}")

# Clean up generated files and directory
source_file_path.unlink()
for item in cache_dir_path.iterdir():
    if item.is_file():
        item.unlink()
cache_dir_path.rmdir()

view raw JSON →