LlamaIndex LLMs Bedrock Integration

Common errors

• botocore.exceptions.NoRegionError: You must specify a region.

  cause The AWS region was not specified during Bedrock LLM initialization or via environment variables (AWS_REGION).
  fix When initializing `Bedrock`, provide the `region_name` argument (e.g., `Bedrock(..., region_name='us-west-2')`) or set the `AWS_REGION` environment variable.

• ValueError: Provider bedrock/mistral for model bedrock/mistral.mixtral-8x7b-instruct-v0:1 is not supported

  cause The model ID string format or the model itself is not recognized by the underlying Bedrock integration layer (e.g., LiteLLM if used indirectly), or the model is not accessible in the specified region.
  fix Verify the exact model ID string from AWS Bedrock documentation (e.g., 'mistral.mixtral-8x7b-instruct-v0:1' or 'amazon.titan-text-express-v1'). Also, ensure the model is enabled and available in your AWS account and the specified `region_name`.

• litellm.APIConnectionError: BedrockException - {"Message":"User: arn:aws:sts::xxxxxxxxxxxxxxxx is not authorized to perform: bedrock:InvokeModel on resource: arn:aws:bedrock:us-west-2::foundation-model/mistral.mixtral-8x7b-instruct-v0:1 with an explicit deny in a service control policy"}

  cause The AWS IAM user or role attempting to invoke the Bedrock model lacks the necessary `bedrock:InvokeModel` permission for the specific model and region.
  fix Review your AWS IAM policies and ensure the user/role has permission to invoke Bedrock models, specifically `bedrock:InvokeModel` on the target resource (e.g., `arn:aws:bedrock:us-west-2::foundation-model/mistral.mixtral-8x7b-instruct-v0:1`). Also, check for any Service Control Policies (SCPs) that might be explicitly denying access.

• TypeError: 'prompt' parameter must be a string for completion, not a list of messages (or vice-versa for chat)

  cause Attempting to use `llm.complete()` with a list of `ChatMessage` objects, or `llm.chat()` with a raw string prompt, when the underlying model or `llama-index` wrapper expects a different type.
  fix For `llm.complete(prompt_string)`, `prompt_string` must be a `str`. For `llm.chat(messages)`, `messages` must be a `list` of `ChatMessage` objects. Convert your input to the expected type for the method being called.

Warnings

• deprecated The `llama-index-llms-bedrock` package is officially deprecated in favor of `llama-index-llms-bedrock-converse`. The `bedrock-converse` package utilizes the newer Bedrock Converse API, which is the recommended way to interact with Bedrock LLMs and provides enhanced capabilities like native function calling.
  Fix: Migrate your code to use `llama-index-llms-bedrock-converse`. Install it with `pip install llama-index-llms-bedrock-converse` and import `BedrockConverse` from `llama_index.llms.bedrock_converse`.
• gotcha AWS region configuration is critical. Ensure the `region_name` parameter passed to `Bedrock` matches the AWS region where your Bedrock model access is granted and where you want to invoke the model. Mismatched regions or an unset region will lead to `NoRegionError` or invocation failures.
  Fix: Explicitly set `region_name` when initializing `Bedrock` (e.g., `region_name='us-east-1'`) or ensure the `AWS_REGION` environment variable is correctly set.
• gotcha Incorrect IAM permissions on your AWS role or user can lead to `BedrockException` indicating 'User is not authorized to perform: bedrock:InvokeModel'.
  Fix: Verify that the IAM role or user credentials used have the necessary `bedrock:InvokeModel` permissions for the specific foundation model and region you are trying to access. Check for any explicit deny policies.
• gotcha For Anthropic Claude models (and potentially others) within `llama-index-llms-bedrock`, there can be prompt formatting differences or type mismatches when using `llm.complete()` versus `llm.chat()`. Specifically, `complete` expects a string prompt, while `chat` expects a list of `ChatMessage` objects. Attempting to pass a list of messages to `complete` (or a string to `chat` without proper wrapping) can result in a `TypeError` or unexpected behavior.
  Fix: Ensure you use `llm.complete(prompt_string)` for single-turn text completions and `llm.chat([ChatMessage(role='user', content='...')])` for multi-turn conversations or when expecting a chat-like interface. Check model-specific documentation for any unique prompt requirements.
• gotcha Passing `guardrail_config` parameters to `Bedrock.complete` might not be correctly processed due to an underlying issue where `kwargs` can be lost. This means Bedrock Guardrails configured this way might not apply to your model invocations.
  Fix: This is a known issue. Consider using `llama-index-llms-bedrock-converse` as it has seen fixes for similar issues regarding parameter passing. Alternatively, implement guardrails at the AWS service level or ensure your `guardrail_config` is being correctly translated and applied in your specific `llama-index` version (check the relevant GitHub issues for updates).

Install

• pip install llama-index-llms-bedrock Install only Bedrock LLM integration
• pip install llama-index llama-index-llms-bedrock boto3 Install with LlamaIndex core and AWS SDK

Imports

• Bedrock

  from llama_index.llms.bedrock import Bedrock

Quickstart

This quickstart demonstrates how to initialize the `Bedrock` LLM, perform a simple text completion, engage in a chat conversation using `ChatMessage` objects, and stream a completion response. Authentication is handled via environment variables for AWS credentials (or a configured AWS CLI profile).

import os
from llama_index.llms.bedrock import Bedrock
from llama_index.core.llms import ChatMessage

# Configure AWS credentials via environment variables for security
# Ensure AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, and AWS_REGION are set
# Or configure AWS CLI profile and use profile_name parameter

# Initialize the Bedrock LLM
llm = Bedrock(
    model=os.environ.get('BEDROCK_MODEL_ID', 'amazon.titan-text-express-v1'),
    region_name=os.environ.get('AWS_REGION', 'us-east-1'),
    temperature=0.7,
    max_tokens=512
)

# Example 1: Simple completion
print("\n--- Completion Example ---")
response_complete = llm.complete("What is the capital of France?")
print(response_complete)

# Example 2: Chat with messages
print("\n--- Chat Example ---")
messages = [
    ChatMessage(role="system", content="You are a helpful AI assistant."),
    ChatMessage(role="user", content="Tell me a short story about a brave knight."),
]
response_chat = llm.chat(messages)
print(response_chat.message.content)

# Example 3: Streaming completion
print("\n--- Streaming Completion Example ---")
print("Streaming response: ", end="")
stream_resp = llm.stream_complete("Explain the concept of AI in simple terms.")
for r in stream_resp:
    print(r.delta, end="")
print("\n")