{"id":7724,"library":"slicerator","title":"Slicerator","description":"Slicerator is a Python library that provides a lazy-loading, fancy-sliceable iterable. It allows you to wrap existing classes or functions to make them behave like reusable generators that support advanced slicing (like NumPy arrays), but only load data when explicitly accessed. The current version is 1.1.0, released in April 2022. While releases are not on a strict schedule, it remains a functional and actively used utility in various projects.","status":"active","version":"1.1.0","language":"en","source_language":"en","source_url":"https://github.com/soft-matter/slicerator","tags":["lazy loading","iterator","slicing","pipeline","data processing"],"install":[{"cmd":"pip install slicerator","lang":"bash","label":"Install with pip"}],"dependencies":[{"reason":"Slicerator requires Python 3.7 or newer to run.","package":"python","optional":false}],"imports":[{"symbol":"Slicerator","correct":"from slicerator import Slicerator"},{"note":"Used as a decorator for lazy function evaluation, often imported alongside Slicerator.","symbol":"pipeline","correct":"from slicerator import pipeline"}],"quickstart":{"code":"from slicerator import Slicerator\n\n@Slicerator.from_class\nclass MyLazyLoader:\n    def __getitem__(self, i):\n        # This method is wrapped by Slicerator to accept slices, lists of integers, or boolean masks.\n        # The code below will only execute when an individual item (integer index) is requested.\n        # In a real application, this would load data from disk, a database, or network.\n        print(f\"[DEBUG] Loading item {i}\")\n        return f\"data_item_{i}\"\n\n    def __len__(self):\n        # Slicerator needs __len__ for proper slicing (e.g., reverse slicing or knowing slice bounds).\n        print(\"[DEBUG] Calculating total length\")\n        return 10\n\n# Instantiate the lazy loader\nloader = MyLazyLoader()\n\n# Create a Slicerator object by slicing. No data is loaded yet.\nsliced_data = loader[::2] # Get every second item\nprint(f\"\\nInitial slice created: {sliced_data}\")\n\n# Further slice the Slicerator. Still no data loaded.\nsub_sliced_data = sliced_data[1:] # Get elements from index 1 onwards of the sliced_data\nprint(f\"Further sliced: {sub_sliced_data}\")\n\n# Access a single item - this triggers loading for that specific item.\nfirst_item = sub_sliced_data[0]\nprint(f\"First item accessed: {first_item}\")\n\n# Convert to list - this triggers loading for all items in `sub_sliced_data`.\nall_items = list(sub_sliced_data)\nprint(f\"All items loaded: {all_items}\")","lang":"python","description":"This quickstart demonstrates how to use the `@Slicerator.from_class` decorator to turn a regular class with `__getitem__` and `__len__` into a lazy-loading, fancy-sliceable iterable. It highlights that actual data loading (`__getitem__` execution) only occurs when specific items are requested or when the Slicerator is fully iterated."},"warnings":[{"fix":"Access specific elements or smaller slices when possible (`my_slicerator[index]`, `my_slicerator[start:end]`) rather than converting the entire object to a collection if lazy loading is desired.","message":"Operations like `list(my_slicerator)` or iterating directly over a Slicerator object (`for item in my_slicerator:`) will force all data to be loaded eagerly, negating the benefits of lazy loading for the entire dataset. Always be mindful of operations that trigger full evaluation.","severity":"gotcha","affected_versions":"All versions"},{"fix":"Consult the official documentation for the order of precedence and recommended practices for attribute propagation. Use explicit methods like the `propagate_attrs` argument in `from_class` for clarity.","message":"The propagation of attributes from the original class to the Slicerator object can be complex. There are multiple mechanisms (`propagate_attrs` parameter, `@propagate_attr` decorator, or a `propagate_attrs` class attribute), and their precedence can lead to unexpected behavior if not understood.","severity":"gotcha","affected_versions":"All versions, specifically 0.9.5 onwards which fixed propagation issues."},{"fix":"Review any code utilizing `slicerator.pipeline` or `Pipeline` objects when upgrading to version 0.9.8 or later. Test pipelines thoroughly to ensure property modifications align with new expectations.","message":"Version 0.9.8 introduced changes enabling `Pipelines` to modify properties. This might alter the expected behavior of existing pipelines that relied on immutable properties or specific side-effects, potentially leading to different outcomes after an upgrade.","severity":"breaking","affected_versions":"Upgrading from <0.9.8 to 0.9.8+"}],"env_vars":null,"last_verified":"2026-04-16T00:00:00.000Z","next_check":"2026-07-15T00:00:00.000Z","problems":[{"fix":"Ensure your class is decorated with `@Slicerator.from_class` or your function is explicitly wrapped with `Slicerator.from_func` to enable fancy slicing. For instance, `loader = MyLazyLoader()` should be `loader = Slicerator.from_class(MyLazyLoader)()` or `loader = MyLazyLoader()` where `MyLazyLoader` is decorated.","cause":"You are attempting to slice a regular Python object (e.g., an instance of `MyLazyLoader`) directly, without it being wrapped or decorated by `Slicerator`.","error":"TypeError: object is not subscriptable"},{"fix":"Implement a `__len__` method in your class that returns the total number of items, or pass the `length` argument when creating a `Slicerator` using `Slicerator.from_func`.","cause":"When using `@Slicerator.from_class` or `Slicerator.from_func`, the underlying object or function is expected to provide a length, typically via a `__len__` method for classes or the `length` argument for `from_func`.","error":"TypeError: object of type 'MyLazyLoader' has no len()"},{"fix":"Use the `propagate_attrs` parameter in `@Slicerator.from_class` (e.g., `@Slicerator.from_class(propagate_attrs=['my_custom_attribute'])`), use the `@propagate_attr` decorator on the method/property within your class, or define a `propagate_attrs` class attribute to explicitly list attributes to be propagated.","cause":"Attributes from the original class are not automatically propagated to the `Slicerator` instance by default. You tried to access an attribute defined on your base class directly on the Slicerator object.","error":"AttributeError: 'Slicerator' object has no attribute 'my_custom_attribute'"}]}