Lance Namespace

0.6.1 · active · verified Thu Apr 09

Lance Namespace is an open specification for describing access and operations against a collection of tables in a multimodal lakehouse, providing a unified model for metadata services and compute engines. The `lance-namespace` Python package offers the core interface and a plugin registry, enabling seamless integration with existing data lakehouse infrastructure, Apache Spark, Ray, and LanceDB for AI and analytics workloads. It is currently at version 0.6.1 and sees active development with frequent releases.

Warnings

breaking The underlying `lance` core library, which `lance-namespace` integrates with, has undergone API cleanups related to namespace handling (e.g., in `lance` 0.6.x). While `lance-namespace` aims for stability, deep integrations might be affected by these changes, especially when upgrading `lance` itself.
Fix: Review the migration guides for the `lance` library for specific breaking changes in namespace-related APIs. Update code to use the latest `lance-namespace` and `lance` API patterns, paying attention to how tables are declared or managed.
gotcha The API is actively evolving with significant new features such as `BatchCommitTables`, `table version operations`, `async client support`, `partition specs`, and `vended credentials` being introduced in recent minor releases (0.4.x - 0.6.x). This rapid evolution can lead to code written for older versions encountering unexpected behavior or requiring updates to leverage new capabilities.
Fix: Always refer to the latest documentation and release notes for the specific version of `lance-namespace` being used. Ensure that code explicitly handles newly introduced features and parameters if applicable, and test thoroughly after upgrades.
gotcha While `lance-namespace` provides the interface, full functionality, especially for native implementations (like directory-based or cloud storage namespaces), often requires the `lance` package (the core Lance data format library) to be installed alongside it. Missing `lance` can lead to errors or limited functionality during connection or table operations.
Fix: Ensure both `lance-namespace` and `lance` are installed: `pip install lance-namespace lance`. Check the specific connector's documentation for any additional requirements.
deprecated The Directory Namespace has two major spec versions: V1 (Directory Listing) and V2 (Manifest). V1 is a simple 1-level namespace suitable for quick starts, while V2 is more advanced, backed by a manifest table, supporting nested namespaces and better performance at scale. Relying on V1 for complex or large-scale deployments may lead to limitations.
Fix: For production or complex use cases, transition to Directory Namespace V2 (Manifest-based) or other supported catalog specifications that offer more robust metadata management. Consult the Lance Namespace documentation for V2 configuration details.

Install

pip install lance-namespace Install latest version

Imports

connect
```
import lance_namespace
ns = lance_namespace.connect(...)
```
The `connect` factory function is the primary entry point for creating namespace instances.

Quickstart

This quickstart demonstrates how to connect to a Lance Namespace using a directory-based implementation. It initializes a `LanceNamespace` instance and lists existing tables. For actual table declaration or data operations, the core `lance` Python package is typically required.

import lance_namespace
import os

# Configure a root path for the directory-based namespace
# For persistent storage, change this to a desired directory or cloud path (e.g., s3://my-bucket/data)
root_path = os.environ.get('LANCE_ROOT_PATH', '/tmp/lance_namespace_data')

# Connect to a directory-based namespace
# This creates a namespace instance that manages tables within the specified directory
try:
    ns = lance_namespace.connect("dir", {"root": root_path})
    print(f"Successfully connected to Lance Namespace at: {root_path}")

    # Example: List tables (initially empty or existing tables)
    tables = ns.list_tables()
    print(f"Tables in namespace: {tables}")

    # In a real scenario, you would then use `ns` to declare, create, or manage Lance tables.
    # For instance, to declare a new table (requires `lance` package for actual data operations):
    # from lance.client import Client
    # client = Client(f"file://{root_path}")
    # table_name = "my_example_table"
    # try:
    #     table_uri = ns.declare_table(table_name, "table_uri_placeholder")
    #     print(f"Declared table '{table_name}' with URI: {table_uri}")
    # except Exception as e:
    #     print(f"Could not declare table '{table_name}': {e}")

except Exception as e:
    print(f"Error connecting to Lance Namespace: {e}")
    print("Ensure the 'lance' package is installed for native implementations and required dependencies are met.")

# Clean up the temporary directory if it was created
# import shutil
# if root_path.startswith('/tmp/') and os.path.exists(root_path):
#     shutil.rmtree(root_path)
#     print(f"Cleaned up temporary directory: {root_path}")

view raw JSON →