{"id":3951,"library":"databricks-ai-bridge","title":"Databricks AI Bridge","description":"Official Python library for Databricks AI support, simplifying Retrieval Augmented Generation (RAG) applications within the Databricks ecosystem. It provides tools for document processing, vectorization, and model serving directly integrated with Databricks services. Currently at version 0.18.0, it follows a pre-1.0 release cadence with frequent updates.","status":"active","version":"0.18.0","language":"en","source_language":"en","source_url":"https://github.com/databricks/databricks-ai-bridge","tags":["databricks","ai","llm","genai","rag","mlops","vector-search"],"install":[{"cmd":"pip install databricks-ai-bridge","lang":"bash","label":"Install core library"}],"dependencies":[{"reason":"Essential for interacting with Databricks APIs and services.","package":"databricks-sdk","optional":false},{"reason":"Core framework for building AI applications, heavily utilized by the library.","package":"langchain","optional":false},{"reason":"Used for MLOps capabilities, particularly model logging and serving integration.","package":"mlflow","optional":false}],"imports":[{"symbol":"DatabricksVectorSearch","correct":"from databricks_ai_bridge.rag.vectorstores import DatabricksVectorSearch"},{"symbol":"PromptEngineer","correct":"from databricks_ai_bridge.model_serving.prompt_engineering import PromptEngineer"},{"symbol":"DatabricksLLM","correct":"from databricks_ai_bridge.model_serving.llm import DatabricksLLM"}],"quickstart":{"code":"import os\nfrom databricks.sdk import WorkspaceClient\nfrom databricks_ai_bridge.rag.vectorstores import DatabricksVectorSearch\nfrom databricks_ai_bridge.model_serving.prompt_engineering import PromptEngineer\nfrom databricks_ai_bridge.model_serving.llm import DatabricksLLM\n\n# Initialize Databricks Workspace Client\n# Ensure DATABRICKS_HOST and DATABRICKS_TOKEN environment variables are set\n# or passed as arguments to WorkspaceClient\nhost = os.environ.get(\"DATABRICKS_HOST\", \"https://dummy-host.cloud.databricks.com\")\ntoken = os.environ.get(\"DATABRICKS_TOKEN\", \"dapi-dummy-token\")\n\nw = None\ntry:\n    # In a real environment, replace dummy values with actual config or ensure env vars are set.\n    w = WorkspaceClient(host=host, token=token)\n    # Attempt to verify client connection (optional, but good for debugging)\n    # w.current_user.me() \n    print(\"Databricks Workspace Client initialized.\")\nexcept Exception as e:\n    print(f\"Warning: Could not initialize Databricks WorkspaceClient. Check DATABRICKS_HOST/TOKEN: {e}\")\n    print(\"Quickstart will proceed with dummy client, but actual interactions will fail.\")\n\nif w:\n    # Example: Using Databricks Vector Search for RAG (requires a configured Vector Search index)\n    # Replace with your actual catalog, schema, table, and index names\n    catalog = \"main\"\n    schema = \"default\"\n    table = \"my_documents_table\"\n    index_name = \"my_vector_search_index\"\n\n    try:\n        vector_store = DatabricksVectorSearch(\n            w,\n            catalog_name=catalog,\n            schema_name=schema,\n            table_name=table,\n            index_name=index_name,\n        )\n        print(f\"Initialized DatabricksVectorSearch for index: {index_name}\")\n\n        # Example: Using PromptEngineer and DatabricksLLM\n        # Ensure a model is served at the specified endpoint in your Databricks workspace\n        llm_endpoint = \"databricks-mixtral-8x7b-instruct\" # Example Foundation Model endpoint or custom served model\n        llm = DatabricksLLM(workspace_client=w, serving_endpoint=llm_endpoint)\n\n        engineer = PromptEngineer(llm=llm, vector_store=vector_store)\n\n        query = \"What is Databricks Lakehouse Platform?\"\n        print(f\"PromptEngineer initialized. A live call to engineer.answer_question('{query}') would retrieve information using the configured LLM and vector store on Databricks.\")\n        # To run live:\n        # response = engineer.answer_question(query)\n        # print(f\"Query: {query}\\nResponse: {response}\")\n\n    except Exception as e:\n        print(f\"Error during quickstart setup. This often means required Databricks resources (e.g., Vector Search index, LLM endpoint) are not configured or accessible: {e}\")\n        print(\"Please ensure your Databricks environment is correctly set up as per databricks-ai-bridge documentation.\")\nelse:\n    print(\"Skipping Databricks AI Bridge component initialization due to WorkspaceClient failure.\")\n","lang":"python","description":"This quickstart demonstrates initializing the Databricks Workspace Client, then setting up DatabricksVectorSearch, DatabricksLLM, and PromptEngineer components. It requires a configured Databricks environment, including environment variables `DATABRICKS_HOST` and `DATABRICKS_TOKEN`, and pre-existing Databricks Vector Search indexes and model serving endpoints for full functionality."},"warnings":[{"fix":"Consult the official `CHANGELOG.md` and updated examples on GitHub for versions 0.17.0+ to adjust pipeline definitions and document loading strategies.","message":"Significant API changes occurred in version 0.17.0, particularly affecting `databricks_ai_bridge.pipelines` and `databricks_ai_bridge.rag.document_loaders`. Users upgrading from older versions (e.g., 0.16.x) will need to refactor their code for these modules.","severity":"breaking","affected_versions":"<0.17.0 (when upgrading to >=0.17.0)"},{"fix":"Ensure `DATABRICKS_HOST` and `DATABRICKS_TOKEN` environment variables are correctly set, and all required Databricks services are configured and accessible in your workspace before running `databricks-ai-bridge` applications.","message":"The library heavily relies on a properly configured Databricks environment, including environment variables (`DATABRICKS_HOST`, `DATABRICKS_TOKEN`) and provisioned Databricks services (Vector Search indexes, Model Serving endpoints, Unity Catalog tables). Operations will fail with `ApiException` or similar errors if these resources are not correctly set up or accessible.","severity":"gotcha","affected_versions":"All versions"},{"fix":"Pin exact minor versions (e.g., `databricks-ai-bridge==0.18.0`) in production environments. Carefully review the `CHANGELOG.md` on GitHub for any new breaking changes when planning an upgrade to a newer minor version.","message":"As a pre-1.0 library, `databricks-ai-bridge` frequently introduces breaking changes in minor versions (e.g., `0.x.0` to `0.x+1.0`). API signatures, module structures, and default behaviors can change without strict adherence to semantic versioning.","severity":"gotcha","affected_versions":"All `0.x.x` versions"}],"env_vars":null,"last_verified":"2026-04-11T00:00:00.000Z","next_check":"2026-07-10T00:00:00.000Z"}