{"id":4500,"library":"delta-kernel-rust-sharing-wrapper","title":"Delta Kernel Rust Sharing Wrapper","description":"This package (PyPI: `delta-kernel-rust-sharing-wrapper`) provides Python bindings to the `delta-kernel-rs` (Rust) crate, enabling Python users to read Delta Lake tables. It exports its functionality via the `delta_kernel_python` module. It serves as a foundational, low-level component, often consumed by higher-level Delta Lake Python clients. Current version is 0.3.1; releases are typically coordinated with updates to the underlying Rust kernel.","status":"active","version":"0.3.1","language":"en","source_language":"en","source_url":"https://github.com/delta-io/delta-kernel-python","tags":["delta-lake","data-lake","rust","ffi","pyarrow","data-reading"],"install":[{"cmd":"pip install delta-kernel-rust-sharing-wrapper","lang":"bash","label":"Install stable version"}],"dependencies":[{"reason":"Required for data interchange with Arrow Dataset API and columnar data processing.","package":"pyarrow","optional":false}],"imports":[{"note":"The PyPI package `delta-kernel-rust-sharing-wrapper` provides the `delta_kernel_python` module for import.","wrong":"import delta_kernel_rust_sharing_wrapper","symbol":"scan_table","correct":"from delta_kernel_python import scan_table"}],"quickstart":{"code":"import delta_kernel_python\nimport pyarrow.dataset as ds\nimport os\n\n# Path to a Delta Lake table.\n# For a runnable example, ensure this path points to a valid Delta table.\n# For cloud paths (s3://, abfss://, etc.), ensure appropriate AWS/Azure credentials\n# are configured (e.g., via environment variables or default credential providers).\n# Example: Create a test table first using `delta-rs` or `pyspark`.\ntable_path = os.environ.get(\"DELTA_TABLE_PATH\", \"./my_local_delta_table\")\n\nprint(f\"Attempting to read Delta table at: {table_path}\")\n\ntry:\n    # Scan the Delta table to get an Arrow Dataset object.\n    # This object can then be used for querying and reading data.\n    dataset: ds.Dataset = delta_kernel_python.scan_table(table_path)\n\n    print(f\"\\nSchema of the Delta table:\\n{dataset.schema}\")\n\n    print(\"\\nFirst 5 records from the table:\")\n    # Read data in batches (e.g., as PyArrow RecordBatches).\n    count = 0\n    for batch in dataset.to_batches():\n        print(batch.to_pylist())\n        count += len(batch)\n        if count >= 5:\n            break\n    if count == 0:\n        print(\"No records found or processed.\")\n\nexcept Exception as e:\n    print(f\"\\nERROR: Could not read Delta table at '{table_path}'.\")\n    print(\"Please ensure the path is correct and points to an existing Delta table.\")\n    print(f\"Original error: {e}\")\n    print(\"Tip: You can create a simple Delta table for testing, e.g., with `delta-rs`:\")\n    print(\"  import pandas as pd\\n  from deltalake import write_deltalake\\n  df = pd.DataFrame({\\\"id\\\": [1, 2], \\\"value\\\": [\\\"A\\\", \\\"B\\\"]})\\n  write_deltalake(\\\"./my_local_delta_table\\\", df, mode=\\\"overwrite\\\")\")","lang":"python","description":"This quickstart demonstrates how to use `delta_kernel_python.scan_table` to read a Delta Lake table and process its data using PyArrow's Dataset API. It assumes a Delta table exists at `DELTA_TABLE_PATH` (or `./my_local_delta_table` by default). Cloud storage paths require correctly configured environment variables for credentials."},"warnings":[{"fix":"Use `import delta_kernel_python` in your Python code.","message":"The PyPI package name (`delta-kernel-rust-sharing-wrapper`) differs from the Python module name (`delta_kernel_python`). Always import using `import delta_kernel_python`.","severity":"gotcha","affected_versions":"All versions"},{"fix":"Ensure your `pyarrow` version falls within the range specified by `pip install delta-kernel-rust-sharing-wrapper` or check the `pyproject.toml` of the specific library version.","message":"This library has specific `pyarrow` version dependencies (e.g., `pyarrow >=10.0.1,<16` for version 0.3.1). Installing an incompatible `pyarrow` version can lead to runtime errors or crashes due to C/Rust FFI mismatches.","severity":"gotcha","affected_versions":"All versions"},{"fix":"Pin your dependency to a specific minor version (e.g., `delta-kernel-rust-sharing-wrapper~=0.3.0`) and review release notes carefully when upgrading.","message":"As this library is pre-1.0.0, its API (`delta_kernel_python` module) is subject to change without strict backward compatibility guarantees. Expect potential breaking changes in minor version updates.","severity":"breaking","affected_versions":"<1.0.0"},{"fix":"Configure cloud storage credentials in your environment or via standard cloud provider SDK configuration methods before running applications that read from cloud-based Delta tables.","message":"When reading tables from cloud storage (e.g., S3, ADLS Gen2), this library relies on the underlying storage client (e.g., `object_store` Rust crate) to pick up credentials. Ensure your environment variables (e.g., `AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY` for S3) or cloud-provider specific configurations are correctly set.","severity":"gotcha","affected_versions":"All versions"}],"env_vars":null,"last_verified":"2026-04-12T00:00:00.000Z","next_check":"2026-07-11T00:00:00.000Z"}