Lightning Utilities

0.15.3 · active · verified Sat Apr 11

Lightning Utilities is a toolbox providing general Python utilities, reusable GitHub workflows, and shared GitHub actions for the broader Lightning ecosystem. It aims to offer common functionalities and development practices across Lightning-AI projects. The current version is 0.15.3, with frequent releases including patch and minor versions every few weeks.

Common errors

```
ImportError: cannot import name 'ModuleAvailableCache' from 'lightning_utilities.core.imports'
```
cause This error occurs when a project, such as older versions of PyTorch Lightning or other dependent libraries, tries to import `ModuleAvailableCache` from `lightning_utilities.core.imports`, but this function has been removed or relocated in the installed `lightning-utilities` version.

fix Downgrade `lightning-utilities` to a compatible version (e.g., `pip install lightning-utilities==0.9.0`) or update all related Lightning ecosystem packages to mutually compatible versions that do not rely on the removed import.
```
ModuleNotFoundError: No module named 'pytorch_lightning.utilities.distributed'
```
cause This specific `ModuleNotFoundError`, often encountered when attempting to import `rank_zero_only`, arises because `pytorch_lightning.utilities.distributed` was refactored. The `rank_zero_only` utility was moved to `pytorch_lightning.utilities.rank_zero` (or similar paths in `lightning_utilities.core.rank_zero`) in PyTorch Lightning versions 1.8.0 and above.

fix Update your import statement to `from pytorch_lightning.utilities.rank_zero import rank_zero_only` or `from lightning_utilities.core.rank_zero import rank_zero_only`. Alternatively, downgrade PyTorch Lightning to a version where the module still existed at the old path (e.g., `pip install pytorch-lightning<1.8.0`).
```
ModuleNotFoundError: No module named 'pkg_resources'
```
cause This error typically indicates that the `setuptools` package, which provides the `pkg_resources` module, is not installed in the environment or is not being correctly resolved. While `lightning-utilities` itself deprecated `pkg_resources` usage in later versions, older installations or specific environments might still encounter this if `setuptools` is missing.

fix Install `setuptools` explicitly using `pip install setuptools`. Ensure your `lightning-utilities` version is `0.15.3` or newer, which addresses `pkg_resources` deprecation warnings and improves compatibility.
```
ModuleNotFoundError: No module named 'pytorch_lightning.utilities.apply_func'
```
cause This error occurs when code attempts to import `apply_func` directly from `pytorch_lightning.utilities`, but its location or the function itself has changed or been deprecated in newer versions of PyTorch Lightning (e.g., after v1.5.0), often moving to `lightning_utilities.core.apply_func` as `apply_to_collection`.

fix Update your import statement to use `from lightning_utilities.core.apply_func import apply_to_collection` and adjust your code to use `apply_to_collection` as needed. If still encountering issues, ensure PyTorch Lightning is updated to its latest stable version.

Warnings

breaking Dropped support for Python 3.9. Users must upgrade to Python 3.10 or newer.
Fix: Upgrade your Python environment to version 3.10 or later.
breaking The internal CLI backend switched from `fire` to `jsonargparse`.
Fix: If you were directly interacting with the `lightning_utilities.cli` module, be aware that argument parsing and internal mechanisms have changed. Consult the `jsonargparse` documentation for updated CLI patterns.
deprecated The `apply_to_collection` utility in `pytorch_lightning.utilities.apply_func` is deprecated.
Fix: Migrate your imports from `pytorch_lightning.utilities.apply_func.apply_to_collection` to `lightning_utilities.core.apply_func.apply_to_collection`.
gotcha Using `apply_to_collection` on frozen dataclasses without explicitly setting `allow_frozen=True` can raise a `MisconfigurationException`.
Fix: If you intend to modify a frozen dataclass using `apply_to_collection`, ensure you pass `allow_frozen=True` to the function call. Otherwise, ensure your dataclass is not frozen or handle the `MisconfigurationException`.
gotcha To use the `lightning_utilities.cli` module, you must install the library with the `[cli]` extra, i.e., `pip install 'lightning-utilities[cli]'`.
Fix: Ensure `jsonargparse` is installed either by installing `lightning-utilities[cli]` or `jsonargparse[signatures]` directly if you intend to use the CLI utilities.

Install

pip install lightning-utilities Default Installation
pip install 'lightning-utilities[cli]' Installation with CLI support

Imports

module_available
```
from lightning_utilities.core.imports import module_available
```
A utility for conditionally importing modules only if they are available.
apply_to_collection
wrong:
```
from pytorch_lightning.utilities.apply_func import apply_to_collection
```
correct:
```
from lightning_utilities.core.apply_func import apply_to_collection
```
The `apply_to_collection` utility has been moved from `pytorch_lightning.utilities` to `lightning_utilities.core` in PyTorch Lightning v1.8.0 and higher.
cli
```
import lightning_utilities.cli
```
Provides common command-line interface (CLI) commands, e.g., for managing requirements.

Quickstart

This example demonstrates the core `apply_to_collection` utility, which recursively applies a function to all elements of a specified data type within a collection.

from lightning_utilities.core.apply_func import apply_to_collection

def double(x):
    return x * 2

data = [1, {'a': 2, 'b': [3, 4]}, (5, 6)]
result = apply_to_collection(data, int, double)
print(result)
# Expected output: [2, {'a': 4, 'b': [6, 8]}, (10, 12)]

view raw JSON →