Docling IBM Models

3.13.0 · active · verified Sat Apr 11

This package provides AI models from IBM, including Large Language Models (LLMs) and embeddings, specifically designed for use with the LangChain framework. It is also an optional dependency for the Docling PDF conversion package, enabling advanced AI capabilities within Docling. Currently at version 3.13.0, it follows a relatively frequent release cadence, with updates typically occurring on a monthly basis.

Warnings

gotcha IBM Cloud API Key, region, and project ID are mandatory for all model interactions with Watsonx.ai services and must be correctly configured. Failure to provide these credentials or providing incorrect ones will result in authentication or authorization errors.
Fix: Set `IBM_CLOUD_API_KEY`, `IBM_CLOUD_REGION`, and `IBM_CLOUD_PROJECT_ID` environment variables, or pass them directly to the `WatsonxLLM` or `WatsonxEmbeddings` constructor. Ensure the API key has necessary permissions for the chosen service and models.
gotcha The availability and compatibility of specific LLM or embedding models (e.g., 'google/flan-ul2', 'intfloat/multilingual-e5-large') can vary by IBM Cloud region and your project's entitlements. Using an unavailable or unauthorized model will result in runtime errors.
Fix: Consult the IBM Watsonx.ai documentation for available models in your target region and ensure your IBM Cloud project has the necessary access permissions. Test with a common, publicly available model first, or verify your model IDs and access through the IBM Cloud console.
breaking When using `docling-ibm-models` with the main `docling` package, be aware that `docling-ibm-models` may pin `docling` to a specific, older version (e.g., `docling==0.1.0`) in its optional dependencies. Installing a newer `docling` version independently can lead to compatibility issues or unexpected behavior.
Fix: If integrating with `docling`, ensure you install `docling` compatible with the version specified in `docling-ibm-models`'s `pyproject.toml`'s optional dependencies. The safest approach is to install `docling-ibm-models` with its `docling` extra: `pip install "docling-ibm-models[docling]"`.
gotcha Depending on the specific models or underlying libraries (e.g., `sentence-transformers` for some embeddings), initial usage might trigger large model downloads, consuming significant disk space and bandwidth. This can impact application startup time and resource usage, especially in containerized environments.
Fix: Ensure adequate disk space and network connectivity for the environment where the models are initialized. Pre-downloading models or configuring cache directories might be necessary for production deployments.

Install

pip install docling-ibm-models Install core package
pip install "docling-ibm-models[docling]" Install with Docling support

Imports

WatsonxLLM
```
from docling_ibm_models.llms import WatsonxLLM
```
Primary class for interacting with IBM Watsonx LLMs.
WatsonxEmbeddings
```
from docling_ibm_models.embeddings import WatsonxEmbeddings
```
Primary class for interacting with IBM Watsonx embedding models.

Quickstart

This quickstart demonstrates how to initialize and use `WatsonxLLM` and `WatsonxEmbeddings` provided by `docling-ibm-models` within the LangChain framework. It requires IBM Cloud API Key, region, and project ID, typically provided via environment variables. The example shows generating text with an LLM and creating embeddings for text.

import os
from docling_ibm_models.llms import WatsonxLLM
from docling_ibm_models.embeddings import WatsonxEmbeddings
from langchain.prompts import PromptTemplate
from langchain_core.output_parsers import StrOutputParser

# --- IBM Cloud Credentials (set as environment variables) ---
# IBM_CLOUD_API_KEY="YOUR_IBM_CLOUD_API_KEY"
# IBM_CLOUD_REGION="us-south" # Example: 'us-south', 'jp-tok', 'eu-de'
# IBM_CLOUD_PROJECT_ID="YOUR_IBM_CLOUD_PROJECT_ID"

api_key = os.environ.get("IBM_CLOUD_API_KEY", "")
region = os.environ.get("IBM_CLOUD_REGION", "us-south")
project_id = os.environ.get("IBM_CLOUD_PROJECT_ID", "")

if not all([api_key, region, project_id]):
    print("Please set IBM_CLOUD_API_KEY, IBM_CLOUD_REGION, and IBM_CLOUD_PROJECT_ID environment variables.")
    print("Skipping LLM and Embeddings quickstart examples.")
else:
    try:
        print("\n--- Initializing WatsonxLLM ---")
        # Initialize the WatsonxLLM
        llm = WatsonxLLM(
            api_key=api_key,
            region=region,
            project_id=project_id,
            model_id="google/flan-ul2", # Example model, check available models for your region
            temperature=0.7,
            max_new_tokens=200
        )

        # Create a simple LangChain chain
        prompt = PromptTemplate.from_template("What is the capital of {country}?")
        output_parser = StrOutputParser()
        chain = prompt | llm | output_parser

        # Invoke the chain
        response = chain.invoke({"country": "France"})
        print(f"Capital of France: {response.strip()}")

        print("\n--- Initializing WatsonxEmbeddings ---")
        # Initialize WatsonxEmbeddings
        embeddings_model = WatsonxEmbeddings(
            api_key=api_key,
            region=region,
            project_id=project_id,
            model_id="intfloat/multilingual-e5-large" # Example embedding model
        )

        # Embed documents
        texts_to_embed = ["Hello world from IBM models", "Docling integration example"]
        text_embeddings = embeddings_model.embed_documents(texts_to_embed)
        print(f"Embeddings for '{texts_to_embed[0]}' (first 5 dims): {text_embeddings[0][:5]}...")

    except Exception as e:
        print(f"\nAn error occurred during quickstart execution: {e}")
        print("Ensure your IBM Cloud credentials are correct, model IDs are valid for your project/region, and you have necessary entitlements.")

view raw JSON →