LlamaIndex Azure OpenAI LLM Integration

0.5.3 · active · verified Mon Apr 13

This integration package allows LlamaIndex to utilize Azure OpenAI's Large Language Model (LLM) services. It provides a robust way to connect to and interact with models like GPT-3.5 and GPT-4 deployed on Azure, leveraging LlamaIndex's indexing and querying capabilities. The current version is 0.5.3, with its release cadence tied to updates in the core LlamaIndex library and Azure OpenAI API changes.

Warnings

breaking LlamaIndex core versions 0.10.0 and above deprecate `ServiceContext` for global configuration. LLMs should now be configured either directly on `Settings.llm` or passed explicitly to constructors.
Fix: Migrate from `ServiceContext(llm=...)` to `from llama_index.core import Settings; Settings.llm = AzureOpenAI(...)`. For specific components, pass the `llm` instance directly.
gotcha The `model` parameter in `AzureOpenAI` refers to your *Azure deployment name*, not the base OpenAI model name (e.g., 'gpt-35-turbo-deployment', not 'gpt-3.5-turbo'). Incorrectly setting this will lead to errors.
Fix: Ensure `model` is set to the exact deployment name as configured in your Azure OpenAI Studio. Also verify `azure_endpoint` is the correct resource endpoint URL and `api_version` matches your deployment's supported version.
gotcha Azure OpenAI API versions (`api_version`) are date-based and specific (e.g., '2023-05-15'). Using an invalid, unsupported, or outdated version can cause API calls to fail or return unexpected results.
Fix: Always use a current and valid `api_version` supported by your Azure OpenAI deployment. Consult Azure's documentation for the latest recommended stable API version. Commonly '2023-05-15' or newer 'preview' versions are used.

Install

pip install llama-index-llms-azure-openai Install package

Imports

AzureOpenAI

from llama_index.llms.azure_openai import AzureOpenAI

Quickstart

This quickstart demonstrates how to initialize the `AzureOpenAI` LLM and use it for a simple completion task. It highlights the required environment variables for Azure OpenAI authentication and configuration, as well as how to set the LLM globally using `Settings` (LlamaIndex v0.10.0+ pattern) or directly interact with the instance.

import os
from llama_index.llms.azure_openai import AzureOpenAI
from llama_index.core import Settings

# Ensure these environment variables are set:
# AZURE_OPENAI_API_KEY
# AZURE_OPENAI_ENDPOINT (e.g., 'https://YOUR_RESOURCE_NAME.openai.azure.com/')
# AZURE_OPENAI_API_VERSION (e.g., '2023-05-15' or '2024-02-15-preview')
# AZURE_OPENAI_LLM_DEPLOYMENT_NAME (your deployment name, e.g., 'gpt-35-turbo-deployment')

azure_openai_api_key = os.environ.get('AZURE_OPENAI_API_KEY', 'YOUR_AZURE_OPENAI_API_KEY')
azure_openai_endpoint = os.environ.get('AZURE_OPENAI_ENDPOINT', 'https://example.openai.azure.com/')
azure_openai_api_version = os.environ.get('AZURE_OPENAI_API_VERSION', '2023-05-15')
azure_llm_deployment_name = os.environ.get('AZURE_OPENAI_LLM_DEPLOYMENT_NAME', 'your-gpt-35-turbo-deployment')

llm = AzureOpenAI(
    model=azure_llm_deployment_name, # This must be your Azure deployment name
    api_key=azure_openai_api_key,
    azure_endpoint=azure_openai_endpoint,
    api_version=azure_openai_api_version,
    temperature=0.7,
)

# Optionally set as default LLM for LlamaIndex global settings
Settings.llm = llm

# Make a completion call
response = llm.complete("Tell me a short story about a brave knight.")
print(response.text)

view raw JSON →