adlfs

2026.2.0 · active · verified Mon Apr 06

adlfs provides an fsspec-compatible interface for accessing Azure Blob Storage, Azure Data Lake Storage Gen2, and the now-deprecated Azure Data Lake Storage Gen1. It allows Python users to interact with Azure storage as if it were a local filesystem, integrating seamlessly with data processing libraries like Dask, Pandas, and Ray. The library is currently at version 2026.2.0 and maintains an active release schedule.

Warnings

breaking ADLS Gen1 (adl:// protocol and AzureDatalakeFileSystem class) has been officially retired. Operations using these interfaces are obsolete.
Fix: Migrate to the `az://` or `abfs://` protocols and use the `adlfs.AzureBlobFileSystem` class for Azure Blob Storage and ADLS Gen2.
breaking Starting in a future release, adlfs will require explicit credentials by default. Anonymous access will no longer be the implicit default.
Fix: If your application relies on anonymous access, explicitly set `anon=True` when creating `adlfs.AzureBlobFileSystem` instances.
gotcha Authentication can be complex due to multiple methods (account key, SAS token, service principal, `DefaultAzureCredential`, connection string). Misconfiguring these or incorrect permissions are common issues.
Fix: Review the `adlfs` and Azure SDK documentation for the specific authentication method you intend to use. Ensure all required credentials are provided and have the necessary permissions for the operations. For `DefaultAzureCredential`, ensure `AZURE_STORAGE_ACCOUNT_NAME` is set and `anon=False`.
breaking Support for Python 3.9 has been removed.
Fix: Upgrade your Python environment to Python 3.10 or newer.
gotcha By default, write operations create BlockBlobs which cannot be appended to. AppendBlobs (using `mode="ab"`) are also not available if hierarchical namespaces are enabled on the storage account.
Fix: Plan your data ingestion strategy around BlockBlobs for most writes. If appending is critical, verify your storage account configuration and consider alternatives if hierarchical namespaces prevent AppendBlobs.

Install

pip install adlfs Install with pip
conda install -c conda-forge adlfs Install with conda

Imports

AzureBlobFileSystem
```
from adlfs import AzureBlobFileSystem
```
This is the primary class for interacting with Azure Blob Storage and ADLS Gen2.
AzureDatalakeFileSystem
```
from adlfs import AzureDatalakeFileSystem
```
The `AzureDatalakeFileSystem` class and `adl://` protocol were for ADLS Gen1, which is now deprecated and removed. For ADLS Gen2, use `AzureBlobFileSystem` with `abfs://` or `az://` protocols.

Quickstart

This quickstart demonstrates how to initialize `adlfs.AzureBlobFileSystem` and list contents of a public Azure Blob Storage container. For private containers, authentication relies on `storage_options` parameters (like `account_key`, `sas_token`, `connection_string`, or service principal details) or automatic credential resolution via `DefaultAzureCredential` (by setting `anon=False` and ensuring `AZURE_STORAGE_ACCOUNT_NAME` is set).

import os
from adlfs import AzureBlobFileSystem

# Recommended: Use environment variables for credentials
# e.g., AZURE_STORAGE_ACCOUNT_NAME, AZURE_STORAGE_ACCOUNT_KEY, AZURE_STORAGE_SAS_TOKEN
# For DefaultAzureCredential, ensure AZURE_STORAGE_ACCOUNT_NAME is set and anon=False

account_name = os.environ.get('AZURE_STORAGE_ACCOUNT_NAME', 'your_account_name')
# For demonstration, using anonymous access to a public container
# In real scenarios, provide proper credentials (account_key, sas_token, or use anon=False)
fs = AzureBlobFileSystem(account_name=account_name, anon=True)

# Example: List contents of a public container
container_name = "azureopendatastorage" # A known public container
path_to_list = f"az://{container_name}/" 

try:
    print(f"Listing contents of {path_to_list}:")
    contents = fs.ls(path_to_list, detail=False)
    for item in contents[:5]: # Print first 5 items
        print(item)
    if not contents: print("Container is empty or access denied (check credentials/permissions).")
except Exception as e:
    print(f"An error occurred: {e}")
    print("Please ensure 'AZURE_STORAGE_ACCOUNT_NAME' is set, or if accessing a private container, provide valid credentials.")

# Example: Read a file (requires appropriate permissions)
# Replace with a real path if you have authenticated access
# file_path = f"az://{container_name}/path/to/your/file.txt"
# try:
#     with fs.open(file_path, 'rb') as f:
#         data = f.read()
#         print(f"\nContent of {file_path[:50]}...: {data.decode()[:100]}...")
# except Exception as e:
#     print(f"\nCould not read file {file_path[:50]}...: {e}")

view raw JSON →