OpenAI LLM Integration for LlamaIndex

0.7.5 · active · verified Thu Apr 09

This package provides the OpenAI Large Language Model (LLM) integration for LlamaIndex (version 0.7.5). LlamaIndex is a data framework designed to connect LLMs with your private or domain-specific data, enabling applications like RAG (Retrieval Augmented Generation). This integration allows LlamaIndex users to leverage various OpenAI models for text completion, chat generation, streaming responses, and structured outputs within their LlamaIndex applications. The library is actively maintained and releases are tied to the broader LlamaIndex ecosystem updates.

Warnings

breaking With LlamaIndex v0.9.x and newer, direct imports from the root `llamaindex` package for LLMs and other integrations have been removed due to modularization. You must now import `OpenAI` directly from `llama_index.llms.openai`.
Fix: Update your import statements from `from llamaindex import OpenAI` to `from llama_index.llms.openai import OpenAI`.
breaking The `ServiceContext` object has been deprecated in LlamaIndex v0.10.x and completely removed in v0.11.x. Global LLM and embedding configurations are now managed through the `Settings` object, and explicit setting of LLMs (like OpenAI) is required.
Fix: Migrate from `ServiceContext` to `Settings`. Instead of `service_context = ServiceContext.from_defaults(llm=OpenAI())`, use `from llama_index.core import Settings; Settings.llm = OpenAI()`. Explicitly set your LLM and embedding model.
gotcha The OpenAI LLM client, by default, expects your OpenAI API key to be set as an environment variable named `OPENAI_API_KEY`. If this variable is not set, you will encounter authentication errors.
Fix: Set the `OPENAI_API_KEY` environment variable before running your application (e.g., `export OPENAI_API_KEY='sk-...'`). Alternatively, pass the API key directly to the `OpenAI` constructor: `llm = OpenAI(api_key='sk-...', model='gpt-3.5-turbo')`.
gotcha While `pip install llama-index` includes a starter bundle with `llama-index-core` and `llama-index-llms-openai`, if you're installing only `llama-index-llms-openai` directly, ensure `llama-index-core` is also installed, as it provides fundamental LlamaIndex abstractions.
Fix: If not installing the full `llama-index` package, explicitly install `pip install llama-index-core llama-index-llms-openai`.
gotcha Although a default model (often `gpt-3.5-turbo`) might be used, it is best practice to explicitly specify the `model` parameter when initializing `OpenAI` to ensure consistency and control, especially when using newer models or specific capabilities.
Fix: Initialize the LLM with the desired model, e.g., `llm = OpenAI(model="gpt-4o")` or `llm = OpenAI(model="gpt-3.5-turbo")`.

Install

pip install llama-index-llms-openai Install latest version

Imports

OpenAI
```
from llama_index.llms.openai import OpenAI
```
Beginning with LlamaIndex 0.9.x, direct imports from specific integration packages are required due to modularization. The older pattern (`from llamaindex import OpenAI`) is no longer supported for new versions.
ChatMessage
```
from llama_index.core.llms import ChatMessage
```
ChatMessage is part of the core LlamaIndex LLM abstractions.

Quickstart

This quickstart demonstrates how to initialize the OpenAI LLM and perform basic text completion and chat interactions. It assumes your OpenAI API key is set as an environment variable (`OPENAI_API_KEY`). You can explicitly pass the `api_key` argument during initialization if preferred. It also shows how to use ChatMessage from `llama_index.core.llms` for chat interactions.

import os
from llama_index.llms.openai import OpenAI

# Ensure your OpenAI API key is set as an environment variable (OPENAI_API_KEY)
# If not, you can pass it directly to the OpenAI constructor:
# llm = OpenAI(api_key="your_api_key_here", model="gpt-3.5-turbo")

openai_api_key = os.environ.get('OPENAI_API_KEY', 'YOUR_OPENAI_API_KEY')
if openai_api_key == 'YOUR_OPENAI_API_KEY':
    print("WARNING: Please set the OPENAI_API_KEY environment variable or replace 'YOUR_OPENAI_API_KEY' with your actual key.")

llm = OpenAI(
    model="gpt-3.5-turbo", 
    api_key=openai_api_key # Uses env var if set, otherwise the placeholder
)

# Example: Generate a completion
resp = llm.complete("Tell me a short story about a brave knight.")
print(f"Completion: {resp}")

# Example: Send a chat message
from llama_index.core.llms import ChatMessage
messages = [
    ChatMessage(role="system", content="You are a helpful assistant."),
    ChatMessage(role="user", content="What is the capital of Canada?")
]
chat_resp = llm.chat(messages)
print(f"Chat Response: {chat_resp.content}")

view raw JSON →