{"id":4637,"library":"morefs","title":"morefs: A Collection of fsspec-based Filesystems","description":"morefs is a Python library offering a collection of self-contained filesystems built on top of `fsspec`. It currently provides `AsyncLocalFileSystem`, `DictFileSystem`, and `OverlayFileSystem` to extend `fsspec`'s capabilities for asynchronous local operations, in-memory dictionary-backed storage, and layered filesystem views, respectively. The library is actively maintained, with the current version being 0.2.2, and releases occur as needed to address compatibility or introduce new features.","status":"active","version":"0.2.2","language":"en","source_language":"en","source_url":"https://github.com/iterative/morefs","tags":["filesystem","fsspec","async","in-memory","overlay"],"install":[{"cmd":"pip install morefs","lang":"bash","label":"Install stable version"}],"dependencies":[{"reason":"Core dependency providing the filesystem abstraction, required for all filesystems.","package":"fsspec","optional":false},{"reason":"Required by `AsyncLocalFileSystem` for its asynchronous operations.","package":"aiohttp","optional":false}],"imports":[{"symbol":"AsyncLocalFileSystem","correct":"from morefs.asyn_local import AsyncLocalFileSystem"},{"symbol":"DictFileSystem","correct":"from morefs.dict import DictFileSystem"},{"symbol":"OverlayFileSystem","correct":"from morefs.overlay import OverlayFileSystem"},{"note":"Used to instantiate morefs filesystems via their protocol string (e.g., 'asynclocal').","symbol":"filesystem","correct":"from fsspec import filesystem"}],"quickstart":{"code":"from fsspec import filesystem\nfrom morefs.asyn_local import AsyncLocalFileSystem\nfrom morefs.dict import DictFileSystem\nfrom morefs.overlay import OverlayFileSystem\nimport asyncio\n\n# AsyncLocalFileSystem\nasync def run_asynclocal_example():\n    fs = filesystem(\"asynclocal\") # or AsyncLocalFileSystem()\n    # Using AsyncLocalFileSystem requires an async context\n    print(f\"AsyncLocalFileSystem ls '/': {await fs.ls('/')}\")\n\n# DictFileSystem\ndict_fs = DictFileSystem({\"foo\": b\"bar\", \"baz/qux\": b\"quux\"})\nprint(f\"DictFileSystem ls '/': {dict_fs.ls('/')}\")\nprint(f\"DictFileSystem read 'foo': {dict_fs.read_bytes('foo')}\")\n\n# OverlayFileSystem\nlower_fs = DictFileSystem({\"foo\": b\"bar_lower\", \"common\": b\"lower_val\"})\nupper_fs = DictFileSystem({\"baz\": b\"qux_upper\", \"common\": b\"upper_val\"})\noverlay_fs = OverlayFileSystem(lower_fs=lower_fs, upper_fs=upper_fs)\nprint(f\"OverlayFileSystem ls '/': {overlay_fs.ls('/')}\")\nprint(f\"OverlayFileSystem read 'foo': {overlay_fs.read_bytes('foo')}\")\nprint(f\"OverlayFileSystem read 'baz': {overlay_fs.read_bytes('baz')}\")\nprint(f\"OverlayFileSystem read 'common' (upper overrides): {overlay_fs.read_bytes('common')}\")\n\n# Run the async example\nasyncio.run(run_asynclocal_example())\n","lang":"python","description":"Demonstrates the initialization and basic usage of `AsyncLocalFileSystem`, `DictFileSystem`, and `OverlayFileSystem`, including listing directories and reading file content."},"warnings":[{"fix":"Ensure `fsspec` is updated to version `2024.5.0` or newer: `pip install --upgrade fsspec`.","message":"Versions of morefs 0.2.1 and later require `fsspec>=2024.5.0` due to API changes in `fsspec`. Using an older version of `fsspec` may lead to runtime errors or unexpected behavior.","severity":"breaking","affected_versions":"0.2.1+"},{"fix":"Verify that existing code using `AsyncLocalFileSystem` works as expected after upgrading to 0.1.0 or later. Update any direct `from morefs.asyncfs import ...` imports to `from morefs.asyn_local import ...`.","message":"The `AsyncLocalFileSystem` underwent a significant internal refactor in version 0.1.0 to wrap `fsspec.LocalFileSystem`. While the public API aimed for compatibility, users relying on specific prior internal behaviors or direct imports from the pre-0.1.0 `asyncfs` module (now `asyn_local`) might experience subtle changes or broken imports.","severity":"gotcha","affected_versions":"<0.1.0 to 0.1.0+"},{"fix":"Pin `morefs` to an exact version in your `requirements.txt` or `pyproject.toml` (e.g., `morefs==0.2.2`) and thoroughly review changelogs before upgrading.","message":"As `morefs` is currently in its 0.x.x version series, its API is still considered somewhat experimental and may undergo non-backward-compatible changes in minor releases. Users should pin exact versions in production environments.","severity":"gotcha","affected_versions":"All 0.x.x versions"}],"env_vars":null,"last_verified":"2026-04-12T00:00:00.000Z","next_check":"2026-07-11T00:00:00.000Z"}