{"id":2696,"library":"pylance","title":"Lance Python SDK","description":"Pylance is the Python SDK for the Lance columnar data format, an open lakehouse format optimized for AI/ML workflows. It offers high-performance vector search, efficient random access, and built-in data versioning and lineage. The library leverages Apache Arrow for data interchange and its core bindings are implemented in Rust via PyO3 for performance. It seamlessly integrates with popular data science tools like Pandas, DuckDB, Polars, PyArrow, and Ray. The current version is 4.0.0, released on March 30, 2026, and the project has a regular release cadence with stable releases approximately every two weeks, although its development status is currently marked as 'Alpha'.","status":"active","version":"4.0.0","language":"en","source_language":"en","source_url":"https://github.com/lance-format/lance","tags":["data format","columnar","ai","machine learning","vector search","lakehouse","apache arrow","data science"],"install":[{"cmd":"pip install pylance","lang":"bash","label":"Install stable version"},{"cmd":"pip install --pre --extra-index-url https://pypi.fury.io/lance-format/ pylance","lang":"bash","label":"Install preview version"}],"dependencies":[{"reason":"Lance is built on Apache Arrow for data interchange.","package":"pyarrow","optional":false},{"reason":"Commonly used for DataFrame creation and manipulation with Lance datasets.","package":"pandas","optional":true},{"reason":"Often used for array operations, particularly with vector data.","package":"numpy","optional":true}],"imports":[{"symbol":"lance","correct":"import lance"}],"quickstart":{"code":"import lance\nimport pandas as pd\nimport numpy as np\n\n# Create a simple Pandas DataFrame\ndf = pd.DataFrame({\n    \"id\": [1, 2, 3],\n    \"vector\": [[0.1, 0.2], [0.3, 0.4], [0.5, 0.6]],\n    \"text\": [\"apple\", \"banana\", \"cherry\"]\n})\n\n# Define a URI for your Lance dataset (local path in this case)\nlance_uri = \"./my_lance_dataset.lance\"\n\n# Write the DataFrame to Lance format\ndataset = lance.write_dataset(df, lance_uri, mode=\"overwrite\")\n\nprint(f\"Lance dataset created at: {lance_uri}\")\n\n# Open the dataset and read it back\nread_dataset = lance.dataset(lance_uri)\nread_df = read_dataset.to_pandas()\n\nprint(\"\\nData read from Lance dataset:\")\nprint(read_df)\n\n# Clean up the created directory (optional)\nimport shutil\nshutil.rmtree(lance_uri)\nprint(f\"\\nCleaned up {lance_uri}\")","lang":"python","description":"This quickstart demonstrates how to create a Lance dataset from a Pandas DataFrame, write it to disk, and then read it back. It also includes cleanup of the created files."},"warnings":[{"fix":"Be prepared for potential API changes and refer to the official documentation for the latest usage patterns.","message":"The library is currently in '3 - Alpha' development status, indicating that the API is not yet stable. Breaking changes may occur between minor versions.","severity":"gotcha","affected_versions":"All versions up to 4.0.0"},{"fix":"Familiarize yourself with cloud storage configurations and performance considerations when working with Lance datasets.","message":"Lance is optimized to be cloud-native, and remote object storage is now the default paradigm. Expect different performance characteristics and potentially require different setup compared to traditional local file-system workflows.","severity":"gotcha","affected_versions":"All versions from 0.3.0 onwards"},{"fix":"Ensure `duckdb` is updated to version 0.7+ if you plan to use it with Lance.","message":"When integrating with DuckDB, specific functionalities may require DuckDB version 0.7 or newer to avoid potential segfaults or unexpected behavior.","severity":"gotcha","affected_versions":"All versions up to 4.0.0"},{"fix":"Use `pip install pylance` for stable releases in production. For preview features, be prepared to update regularly or adjust dependencies.","message":"While preview releases offer the latest features and bug fixes and are tested similarly to stable releases, they are not guaranteed to be available for more than six months. For production environments, it is recommended to pin to stable releases.","severity":"gotcha","affected_versions":"All versions, particularly preview releases"}],"env_vars":null,"last_verified":"2026-04-10T00:00:00.000Z","next_check":"2026-07-09T00:00:00.000Z"}