{"id":7726,"library":"snakemake-interface-storage-plugins","title":"Snakemake Storage Plugin Interface","description":"This package provides a stable and consistent interface for developers to build custom storage plugins that integrate with Snakemake. It defines the abstract base classes and helper functions necessary for Snakemake to interact with various storage backends. The current version is 4.4.1, with a release cadence of roughly every 1-3 months, primarily focusing on new features, bug fixes, and compatibility with Snakemake core.","status":"active","version":"4.4.1","language":"en","source_language":"en","source_url":"https://github.com/snakemake/snakemake-interface-storage-plugins","tags":["snakemake","workflow","storage","plugin","interface","bioinformatics"],"install":[{"cmd":"pip install snakemake-interface-storage-plugins","lang":"bash","label":"Install latest version"}],"dependencies":[{"reason":"Used for human-readable file sizes and other utilities within the interface. It became a direct dependency from v4.3.3 onwards.","package":"humanfriendly","optional":false}],"imports":[{"symbol":"StorageProviderBase","correct":"from snakemake_interface_storage_plugins.storage_provider import StorageProviderBase"},{"symbol":"StorageObject","correct":"from snakemake_interface_storage_plugins.storage_object import StorageObject"},{"symbol":"StorageProviderSettingsBase","correct":"from snakemake_interface_storage_plugins.settings import StorageProviderSettingsBase"},{"note":"The StoragePlugin class for registering plugins was moved to the 'registry' submodule.","wrong":"from snakemake_interface_storage_plugins.plugin import StoragePlugin","symbol":"StoragePlugin","correct":"from snakemake_interface_storage_plugins.registry import StoragePlugin"}],"quickstart":{"code":"from snakemake_interface_storage_plugins.storage_object import StorageObject\nfrom snakemake_interface_storage_plugins.storage_provider import StorageProviderBase\nfrom snakemake_interface_storage_plugins.registry import StoragePlugin\nfrom typing import Optional\n\n# 1. Define your custom StorageObject, implementing required methods\nclass MyMinimalStorageObject(StorageObject):\n    def exists(self) -> bool:\n        # Placeholder: In a real plugin, check if the remote object exists.\n        return True\n\n    def mtime(self) -> float: return 0.0 # Last modified time\n    def size(self) -> int: return 0      # Size in bytes\n    def download_obj(self) -> None: pass # Download object to local path\n    def upload_obj(self) -> None: pass   # Upload local object to remote\n    def list_local_files(self) -> list[str]: return [] # List files in local path\n    def cleanup(self) -> None: pass      # Cleanup temporary files\n\n# 2. Define your custom StorageProvider\nclass MyMinimalStorageProvider(StorageProviderBase):\n    def get_storage_object(self, url: str) -> StorageObject:\n        # Return an instance of your custom StorageObject for the given URL.\n        # 'query' and 'protocol' are typically parsed from the URL.\n        return MyMinimalStorageObject(query=None, protocol=self.name, path=url)\n\n    def example_path(self, protocol: Optional[str] = None) -> str:\n        # Provide an example path for your protocol (e.g., for documentation).\n        return f\"{self.name}://path/to/file.txt\"\n\n    @property\n    def name(self) -> str:\n        # The unique name for your storage protocol (e.g., 's3', 'gcs', 'my-minimal').\n        return \"my-minimal\"\n\n# 3. Register your plugin with Snakemake\n# This makes your plugin discoverable by Snakemake when it parses Snakefiles.\nStoragePlugin(\n    name=\"my-minimal\",\n    storage_provider=MyMinimalStorageProvider,\n    # settings_cls=None # Optionally register a custom settings class\n)\n\nprint(\"Minimal Snakemake storage plugin 'my-minimal' defined and registered.\")\n# To make this plugin active for Snakemake, save this code in a Python file\n# (e.g., my_storage_plugin.py) and ensure it's on your PYTHONPATH or installed\n# as part of a Snakemake extension package.","lang":"python","description":"This quickstart demonstrates the core components required to implement a minimal Snakemake storage plugin. You need to define a `StorageObject` (representing a file/directory on your custom storage) and a `StorageProviderBase` (which provides `StorageObject` instances). Finally, register your plugin using `StoragePlugin` to make it discoverable by Snakemake."},"warnings":[{"fix":"Review `IOCacheInterface` and `StorageObject` definitions. Implement checksum handling methods/attributes in your custom `StorageObject` and `IOCacheInterface` (if used) to avoid potential future compatibility issues or `TypeError`s when Snakemake expects this functionality.","message":"The `checksum` attribute was added to `IOCacheInterface` and `StorageObject` in v4.4.0. Existing custom storage plugins might need to be updated to properly handle checksum generation and validation to maintain full compatibility and leverage new Snakemake features.","severity":"breaking","affected_versions":">=4.4.0"},{"fix":"Ensure `humanfriendly` is installed in your environment: `pip install humanfriendly`. Upgrade to version 4.3.3 or newer to have it automatically included as a dependency.","message":"Older versions of `snakemake-interface-storage-plugins` (prior to v4.3.3) did not explicitly list `humanfriendly` as a direct dependency. If `humanfriendly` is not installed by another package, you might encounter a `ModuleNotFoundError`.","severity":"gotcha","affected_versions":"<4.3.3"},{"fix":"Review how your custom plugin handles file/directory not found scenarios and other I/O errors. Ensure that custom exceptions are handled or converted appropriately to align with the interface's expected error types, especially `WorkflowError` for user-facing issues.","message":"Changes in error handling, specifically around `FileOrDirectoryNotFoundError` and its conversion to `WorkflowError`, were introduced in versions 4.3.0 and 4.3.1. Custom plugins raising specific exceptions might need adjustment.","severity":"breaking","affected_versions":">=4.3.0"},{"fix":"Upgrade `snakemake-interface-storage-plugins` to v4.4.1 or higher when using Python 3.10 to ensure full compatibility: `pip install --upgrade snakemake-interface-storage-plugins`.","message":"Python 3.10 compatibility fixes were applied in v4.4.1. If you are using Python 3.10 with versions prior to 4.4.1, you might encounter unexpected runtime errors or incorrect behavior.","severity":"gotcha","affected_versions":"<4.4.1"}],"env_vars":null,"last_verified":"2026-04-16T00:00:00.000Z","next_check":"2026-07-15T00:00:00.000Z","problems":[{"fix":"Install the missing dependency: `pip install humanfriendly`. Alternatively, upgrade to `snakemake-interface-storage-plugins>=4.3.3` which includes `humanfriendly` as a direct dependency.","cause":"The `humanfriendly` package, used internally by the storage plugin interface, was not installed as a direct dependency in versions prior to v4.3.3.","error":"ModuleNotFoundError: No module named 'humanfriendly'"},{"fix":"Ensure that your `MyCustomStorageObject` class (and any custom `IOCacheInterface` if used) implements all abstract methods defined in the `StorageObject` base class. You must provide concrete implementations for `exists`, `mtime`, `size`, `download_obj`, `upload_obj`, `list_local_files`, and `cleanup`.","cause":"Your custom `StorageObject` class inherits from `snakemake_interface_storage_plugins.storage_object.StorageObject` but has not implemented all of its required abstract methods.","error":"TypeError: Can't instantiate abstract class MyCustomStorageObject with abstract methods exists, mtime, size, download_obj, upload_obj, list_local_files, cleanup"},{"fix":"Update your code to provide a `checksum` argument when initializing `StorageObject` or `IOCacheInterface` instances, or ensure your custom plugin adheres to the updated interface. For example: `MyStorageObject(query=None, protocol='my', path=url, checksum=None)` (if checksum is optional for your specific case, otherwise provide a valid checksum value).","cause":"You are trying to instantiate `StorageObject` or `IOCacheInterface` (or a custom class inheriting from them) without providing the `checksum` argument, which became required from v4.4.0 onwards for full functionality.","error":"TypeError: __init__() missing 1 required positional argument: 'checksum'"}]}