etils - Eclectic Python Utilities

1.14.0 · active · verified Thu Apr 09

etils (eclectic utils) is an open-source collection of Python utilities designed for machine learning and scientific computing workflows. Each top-level submodule is self-contained and independent, prefixed by 'e' (e.g., `epath`, `epy`) to avoid name collisions. This modular design allows users to import only the necessary components, minimizing dependencies. The library is actively maintained, with frequent updates.

Warnings

gotcha Etils is designed with independent submodules; importing `etils` directly does not expose all functionalities. Users must explicitly `from etils import <submodule>` (e.g., `from etils import epath`) to access specific utilities.
Fix: Always use `from etils import <submodule>` for required functionality. Do not expect `import etils` to make all sub-utilities available.
breaking Many features are behind optional dependencies (extras). Installing only `pip install etils` will lead to `ModuleNotFoundError` if you attempt to use functionality from submodules like `epath` or `ejax` without installing `pip install etils[epath]` or similar.
Fix: Install `etils` with the specific extras corresponding to the submodules you intend to use, e.g., `pip install etils[epath,epy]`.
gotcha The return value format of `epath.Path.stat()` might not strictly match `pathlib.Path.stat()`. This can cause subtle incompatibilities or `AttributeError` if code expects specific attributes only present in `pathlib`'s `stat` result.
Fix: Carefully review the documentation for `epath.Path.stat()` and adapt code to its specific return structure, or use only commonly available attributes for maximum compatibility.
deprecated Previous versions may have included or exposed certain utilities (e.g., `epy.cached_property`) that have since been removed or relocated, leading to `AttributeError` on upgrade.
Fix: Consult the `etils` release notes or GitHub issues for migration guides. For `epy.cached_property`, consider using standard library `functools.cached_property` if applicable to your Python version.
gotcha There's an open issue (#756) regarding `eapp.better_logging()` potentially causing double logging output. This can lead to redundant or confusing log messages in applications using `eapp`.
Fix: Monitor GitHub issues for a resolution. As a workaround, inspect your logging configuration to prevent duplicate handlers or adjust `eapp.better_logging()` parameters if available.

Install

pip install etils Base installation (minimal dependencies)
pip install etils[array_types,epath,epy,etree] Installation with common optional modules

Imports

epath
```
from etils import epath
```
Path utilities with cloud storage support (gs://, s3://)
epy
```
from etils import epy
```
Collection of generic Python utilities
etree
```
from etils import etree
```
Tree utilities for tf.nest, jax.tree_utils, DeepMind tree
enp
```
from etils import enp
```
NumPy utilities
ecolab
```
from etils import ecolab
```
Colab/Jupyter utilities
edc
```
from etils import edc
```
Dataclass utilities
eapp
```
from etils import eapp
```
Absl flags/app utilities
array_types
```
from etils import array_types
```
Typing annotations for JAX, NumPy, and other arrays

Quickstart

This example demonstrates the usage of `epath` for basic file operations. `epath.Path` extends `pathlib.Path` to seamlessly handle both local and cloud-based file paths (e.g., `gs://`, `s3://`) by integrating with respective cloud storage libraries.

from etils import epath

# epath provides a pathlib-like API with cloud storage support (gs://, s3://)
# For local paths, it behaves like pathlib.Path
local_path = epath.Path("/tmp/etils_example.txt")
local_path.write_text("Hello from etils epath!")
print(f"File created at: {local_path}")
print(f"Content: {local_path.read_text()}")

# Clean up (optional)
local_path.unlink()

view raw JSON →