Flexcache: Disk-based object caching

Common errors

• ModuleNotFoundError: No module named 'flexcache'

  cause The 'flexcache' library has not been installed in your Python environment or the environment where your script is being run.
  fix Install the library using pip: `pip install flexcache`

• from flexcache import DiskCache

  cause This is a common search query when developers are looking for the correct way to import core components, or encountering an issue where `DiskCache` might not be directly available or named differently in their installed version, leading to an ImportError or AttributeError if they guess incorrectly.
  fix The correct import for the main disk caching class is `from flexcache import DiskCache` or `from flexcache import DiskCacheByMTime` / `DiskCacheByHash` for specific invalidation strategies. Ensure `flexcache` is installed.

• TypeError: Missing 1 required positional argument: 'func'

  cause This error typically occurs when a caching decorator or function expects a callable (the function to be cached) but does not receive it, often due to incorrect application of the decorator or direct function call.
  fix Ensure the `@cache_decorator` (or equivalent) is correctly applied directly above a function definition, or that you are passing a callable function as the required argument when calling a caching utility directly.

• FileNotFoundError: [Errno 2] No such file or directory: '/path/to/cache/directory'

  cause The specified cache directory either does not exist or the Python process lacks the necessary permissions to create or access it.
  fix Ensure the `cache_dir` argument (or equivalent configuration) points to an existing and writeable directory, or that the Python script has permissions to create the directory if it doesn't exist.

Warnings

• gotcha The library uses Python's `pickle` protocol for serialization of cached objects. Deserializing data from an untrusted source using `pickle` can lead to arbitrary code execution. Ensure that cached files are only loaded from trusted sources or implement custom, secure serialization.
  Fix: Only use `flexcache` with trusted input sources for cached data. For sensitive applications, consider implementing a custom cache (subclassing `DiskCache`) with a safer serialization method like JSON or a custom binary format for trusted, known data types.
• gotcha Choosing the correct cache invalidation strategy (`DiskCacheByMTime` vs. `DiskCacheByHash`) is crucial. `DiskCacheByMTime` relies on file modification timestamps, which might not detect changes if a file's content is altered without updating its timestamp (e.g., atomic writes that replace the file).
  Fix: If the integrity of cached data is paramount and source files might change without MTime updates, use `DiskCacheByHash`. Otherwise, `DiskCacheByMTime` offers better performance for frequently updated files.
• breaking As `flexcache` is in a pre-1.0 version (0.3), its API and internal behavior may not be fully stable. Future minor releases (e.g., 0.4, 0.5) might introduce breaking changes without strict adherence to semantic versioning until a 1.0 release.
  Fix: Pin the `flexcache` version in your `requirements.txt` (e.g., `flexcache==0.3.0`) and review release notes thoroughly before upgrading to any new minor version.

Install

• pip install flexcache Install with pip

Imports

• DiskCacheByMTime

  from flexcache import DiskCacheByMTime

  Commonly used for caching where invalidation is based on file modification time.
• DiskCacheByHash

  from flexcache import DiskCacheByHash

  Used when cache invalidation requires detecting changes via file content hash, offering stronger integrity checks than modification time.
• DiskCache

  from flexcache import DiskCache

  Base class for implementing custom disk caching logic.

Quickstart

This quickstart demonstrates how to initialize a `DiskCacheByMTime` instance, define an expensive 'reader' function, and use the `load` method to automatically cache and retrieve transformed data. It also shows how modifications to the source object invalidate the cache, triggering a re-execution of the reader. Ensure `os` is imported for the quickstart to correctly handle environment variables for cache directory paths.

import pathlib
from flexcache import DiskCacheByMTime

# Create a cache directory (ensure it exists)
cache_dir_path = pathlib.Path(os.environ.get('FLEXCACHE_CACHE_DIR', './my_flexcache_cache'))
cache_dir_path.mkdir(parents=True, exist_ok=True)

dc = DiskCacheByMTime(cache_folder=cache_dir_path)

def expensive_parser(source_path: pathlib.Path) -> str:
    """Simulates an expensive operation that reads and transforms a file."""
    print(f"[Parser] Reading and processing: {source_path}")
    return source_path.read_text().upper()

# Create a dummy source file
source_file_path = pathlib.Path("source.txt")
source_file_path.write_text("Hello, Flexcache World!")

# First call: `expensive_parser` will be executed and result cached
print("\n--- First call ---")
parsed_content_1 = dc.load(source_file_path, reader=expensive_parser)
print(f"Result: {parsed_content_1}")

# Second call: result will be loaded from cache without executing `expensive_parser`
print("\n--- Second call (from cache) ---")
parsed_content_2 = dc.load(source_file_path, reader=expensive_parser)
print(f"Result: {parsed_content_2}")

# Modify the source file to invalidate the cache (for DiskCacheByMTime)
source_file_path.write_text("UPDATED content for Flexcache!")

# Third call: `expensive_parser` will be re-executed due to source modification
print("\n--- Third call (cache invalidated) ---")
parsed_content_3 = dc.load(source_file_path, reader=expensive_parser)
print(f"Result: {parsed_content_3}")

# Clean up generated files and directory
source_file_path.unlink()
for item in cache_dir_path.iterdir():
    if item.is_file():
        item.unlink()
cache_dir_path.rmdir()