LlamaIndex OpenAI Question Generator

Warnings

• gotcha The `OpenAIQuestionGenerator` is specifically designed for OpenAI models that support the function calling API (e.g., `gpt-3.5-turbo-0613`, `gpt-4`). It will not work with older OpenAI completion-based models or other generic LLMs that do not support this API, which are typically handled by `LLMQuestionGenerator`.
  Fix: Ensure you are using a compatible OpenAI model (e.g., `gpt-3.5-turbo-0613` or newer) when initializing `OpenAIQuestionGenerator`.
• breaking As of LlamaIndex v0.10.x, the library adopted a modular, namespaced package structure. This means integration packages like `llama-index-question-gen-openai` must be installed explicitly. Importing components directly from `llama_index` or `llama_index.core` for integrations that have moved to separate packages will result in `ImportError` if the specific integration package is not installed.
  Fix: Always `pip install` the specific `llama-index-*-*` integration package you intend to use (e.g., `pip install llama-index-question-gen-openai`).
• gotcha An `OPENAI_API_KEY` must be set as an environment variable (e.g., `export OPENAI_API_KEY='sk-...'` or via a `.env` file) for all OpenAI integrations, including `OpenAIQuestionGenerator`. Failure to do so will result in authentication errors when making API calls.
  Fix: Before running your application, ensure `OPENAI_API_KEY` is properly configured in your environment. For quick testing, you can directly set `os.environ["OPENAI_API_KEY"]`.
• gotcha When using `OpenAIQuestionGenerator` within a `SubQuestionQueryEngine` with multiple tools, the combined descriptions of these tools can exceed OpenAI's function calling API character limit (currently 1024 characters). This will raise a `ValueError`.
  Fix: Shorten tool descriptions or consider moving extensive details into the prompt itself rather than solely relying on tool metadata. Evaluate if fewer, more broadly defined tools can achieve the same goal without hitting the limit.
• gotcha Network issues, incorrect API keys, or exceeding rate limits can lead to `APIConnectionError` or `RateLimitError` when calling the OpenAI API. These are common with external API interactions and can disrupt workflows.
  Fix: Implement robust error handling with `try-except` blocks around API calls. Consider using a retry mechanism with exponential backoff (e.g., via the `tenacity` library) to manage transient network issues and rate limits. Verify your API key and monitor usage in your OpenAI dashboard.

Install

• pip install llama-index-question-gen-openai Install package

Imports

• OpenAIQuestionGenerator

  from llama_index.question_gen.openai import OpenAIQuestionGenerator

  Following the LlamaIndex v0.10+ modularization, integrations are in their own namespaced packages, not directly under `llama_index.question_gen`.

Quickstart

This quickstart demonstrates how to initialize `OpenAIQuestionGenerator` and use it within a LlamaIndex application to generate sub-questions. It requires an `OPENAI_API_KEY` and sets up a basic `QueryEngineTool` that the generator can reference. The generated sub-questions are tailored to the provided tools and original query.

import os
from llama_index.question_gen.openai import OpenAIQuestionGenerator
from llama_index.core.tools import ToolMetadata, QueryEngineTool
from llama_index.core import QueryBundle, VectorStoreIndex, SimpleDirectoryReader
from llama_index.llms.openai import OpenAI

# Set your OpenAI API key (replace with your actual key or load from .env)
os.environ["OPENAI_API_KEY"] = os.environ.get("OPENAI_API_KEY", "YOUR_OPENAI_API_KEY")

# Create a dummy data file for demonstration
with open("data.txt", "w") as f:
    f.write("The capital of France is Paris. Paris is known for its Eiffel Tower.")
    f.write("The capital of Germany is Berlin. Berlin has a rich history.")

# Load data and create a simple index for a tool
reader = SimpleDirectoryReader("./")
documents = reader.load_data()
index = VectorStoreIndex.from_documents(documents)
query_engine = index.as_query_engine()

# Define a tool for the question generator to use
tool = QueryEngineTool(
    query_engine=query_engine,
    metadata=ToolMetadata(
        name="city_info",
        description="Provides information about cities and their landmarks."
    ),
)

# Initialize OpenAIQuestionGenerator
# It uses OpenAI's function calling API by default.
question_gen = OpenAIQuestionGenerator.from_defaults(llm=OpenAI(model="gpt-3.5-turbo-0613"))

# Generate sub-questions based on a complex query and available tools
query_bundle = QueryBundle("Tell me about the capitals of European countries and their famous landmarks.")
sub_questions = question_gen.generate(
    tools=[tool],
    query=query_bundle
)

print(f"Generated {len(sub_questions)} sub-questions:")
for sq in sub_questions:
    print(f"- Question: {sq.sub_question}, Tool: {sq.tool_name}")