Google GenAI LLM Integration for LlamaIndex

0.9.1 · active · verified Thu Apr 16

This library provides the official integration for connecting LlamaIndex applications with Google's Generative AI models, including Gemini and Vertex AI. It allows LlamaIndex to leverage Google's LLMs for tasks such as text completion, chat interactions, function calling, and structured prediction. As of version 0.9.1, it is actively maintained with a regular release cadence, often aligning with updates to the core LlamaIndex framework.

Common errors

```
ImportError: cannot import name 'Gemini' from 'llama_index.llms' (unknown location)
```
cause Attempting to import the `Gemini` class from the old, deprecated `llama_index.llms.gemini` path instead of the new `llama_index.llms.google_genai` path.

fix Update your import statement to `from llama_index.llms.google_genai import GoogleGenAI` and ensure `llama-index-llms-google-genai` is installed.
```
google.api_core.exceptions.PermissionDenied: 403 Permission denied. Your application has insufficient authentication credentials to access this API. Please check the 'Authentication' section in the API documentation.
```
cause The `GOOGLE_API_KEY` environment variable is either not set, incorrect, or expired. Or, for Vertex AI, the `project_id` or `location` (region) are not correctly configured.

fix Set the `GOOGLE_API_KEY` environment variable with a valid API key obtained from Google AI Studio. For Vertex AI, verify that `GOOGLE_GENAI_USE_VERTEXAI=True` and that `project_id` and `location` are correctly specified.
```
google.api_core.exceptions.ResourceExhausted: 429 Resource has been exhausted (e.g. check your plan and billing).
```
cause You have exceeded the allowed API request rate limit or quota for Google Generative AI models.

fix Implement a retry mechanism with exponential backoff in your application to handle transient rate limit errors. Review your Google Cloud project's API quotas and consider requesting an increase if your usage consistently exceeds limits.

Warnings

breaking The `llama-index-llms-gemini` package has been discontinued since version 0.6.2 and replaced by `llama-index-llms-google-genai`. Direct imports from `llama_index.llms.gemini` will fail.
Fix: Migrate your code to use the `llama-index-llms-google-genai` package and update import statements to `from llama_index.llms.google_genai import GoogleGenAI`.
gotcha Authentication requires setting the `GOOGLE_API_KEY` environment variable. For Vertex AI, additional configurations like `GOOGLE_GENAI_USE_VERTEXAI`, `region`, and `project_id` (either as environment variables or `vertexai_config` parameters) are necessary. Incorrect setup will lead to authentication errors.
Fix: Ensure `GOOGLE_API_KEY` is correctly set in your environment. For Vertex AI, configure `GOOGLE_GENAI_USE_VERTEXAI=True` and provide `project_id` and `location` (region) either via environment variables or in the `GoogleGenAI` constructor.
gotcha API rate limiting from Google GenAI can lead to `google.api_core.exceptions.ResourceExhausted` (HTTP 429) errors.
Fix: Implement retry logic with exponential backoff around your LLM calls. Consider checking and potentially increasing your Google Cloud project quotas if these errors persist.
gotcha Token usage metrics (promptTokens, completionTokens) might not be accurately populated or reported in some tracing and observability tools (e.g., Langfuse) for synchronous calls due to how the underlying Google GenAI SDK reports usage metadata.
Fix: Check the documentation of your specific observability tool for updates or workarounds. This is often an issue with the integration layer rather than the LLM integration itself. Manual parsing of `usage_metadata` might be required in some cases.

Install

pip install llama-index-llms-google-genai Install only Google GenAI LLM integration
pip install llama-index google-generativeai llama-index-llms-google-genai Install with LlamaIndex core and Google Generative AI SDK

Imports

GoogleGenAI
wrong:
```
from llama_index.llms.gemini import Gemini
```
correct:
```
from llama_index.llms.google_genai import GoogleGenAI
```
The 'llama-index-llms-gemini' package has been discontinued and replaced. Always use 'llama-index-llms-google-genai'.

Quickstart

This quickstart demonstrates how to initialize the GoogleGenAI LLM and perform a basic text completion. It assumes the `GOOGLE_API_KEY` environment variable is set. Replace `YOUR_GOOGLE_API_KEY_HERE` with your actual key for testing if not using environment variables.

import os
from llama_index.llms.google_genai import GoogleGenAI

# Ensure your Google API key is set as an environment variable
# Or pass it directly: api_key="YOUR_API_KEY"
os.environ["GOOGLE_API_KEY"] = os.environ.get('GOOGLE_API_KEY', 'YOUR_GOOGLE_API_KEY_HERE')

llm = GoogleGenAI(model="gemini-pro") # Or use 'gemini-1.5-flash', 'gemini-1.5-pro', etc.

response = llm.complete("What is the capital of France?")
print(response)

view raw JSON →