Roboflow Inference Python SDK

Common errors

• ModuleNotFoundError: No module named 'torch'

  cause You are attempting to use the `inference-gpu` package, but PyTorch is not installed or not correctly linked for your environment.
  fix Install PyTorch and torchvision with the correct CUDA version for your system: `pip install torch torchvision --index-url https://download.pytorch.org/whl/cuXX` (replace `cuXX` with your CUDA version, e.g., `cu121`), then `pip install inference-gpu`.

• inference_sdk.http.errors.InferenceError: The model failed to load

  cause The specified model_id is incorrect, the model is not accessible (e.g., private model without authentication), or there are environment constraints (e.g., GPU memory limits) that prevent the model from loading on the inference server.
  fix Double-check your `model_id` for typos, ensure your `ROBOFLOW_API_KEY` is correct and has access to the model, and verify that your inference environment meets the model's requirements (e.g., sufficient RAM, GPU memory).

• KeyError: 'predictions'

  cause The inference response object structure might have changed, or your code expects a key that is no longer present, possibly due to a major version update or an error in the inference call itself.
  fix Ensure you are accessing results using the documented properties (e.g., `result.predictions`). Consult the official documentation for the response object structure for your `inference-sdk` version. If using an older `inference-sdk` version, upgrade to the latest stable release.

Warnings

• breaking As of `v1.2.0`, `inference-models` became the default backend for running predictions. While the old backend is available via an opt-out mechanism, this change may affect performance characteristics, model loading behavior, or specific model compatibility.
  Fix: Review your application's inference performance and behavior after updating. If issues arise, consult the documentation for opting out of `inference-models` or adapting your model usage.
• deprecated Python 3.9 reached End of Life (EOL) and support was officially deprecated in `inference-sdk` `v1.1.0`. While it might still function, future updates may break compatibility.
  Fix: Upgrade your Python environment to 3.10, 3.11, or 3.12 to ensure full compatibility and receive future updates.
• gotcha For GPU acceleration, installing `inference-gpu` directly without specific PyTorch/torchvision versions can lead to issues. It's recommended to pre-install PyTorch and torchvision with your desired CUDA version first.
  Fix: Install PyTorch and torchvision with CUDA support *before* installing `inference-gpu`. Example: `pip install torch torchvision --index-url https://download.pytorch.org/whl/cu121` (replace `cu121` with your CUDA version), then `pip install inference-gpu`.

Install

• pip install inference-sdk Standard installation (CPU)
• pip install inference-gpu For GPU acceleration (requires pre-installed PyTorch and torchvision with CUDA support)

Imports

• InferenceHTTPClient

  from inference_sdk import InferenceHTTPClient

• InferenceConfiguration

  from inference_sdk import InferenceConfiguration

Quickstart

This quickstart demonstrates how to initialize the Inference HTTP client, set your API key (preferably via environment variable), and perform object detection inference on an image from a URL using a specified Roboflow model ID.

import os
from inference_sdk import InferenceHTTPClient

# Set your Roboflow API Key as an environment variable:
# export ROBOFLOW_API_KEY="YOUR_API_KEY"
api_key = os.environ.get('ROBOFLOW_API_KEY', '')

if not api_key:
    print("Warning: ROBOFLOW_API_KEY environment variable not set. Inference might fail.")

# Initialize the InferenceHTTPClient
# Use "https://detect.roboflow.com" for hosted inference
# Use "http://localhost:9001" if you're running a local inference server
client = InferenceHTTPClient(api_url="https://detect.roboflow.com", api_key=api_key)

# Specify your Roboflow model ID (e.g., "my-project/1")
# Replace with a real public model or your own private model ID.
model_id = "lego-brick-detector/1" # Example public model

# Define the image source (can be a URL, local path, or base64 string)
image_url = "https://media.roboflow.com/example_input.jpg"

print(f"Attempting inference on {image_url} using model {model_id}...")

try:
    # Perform inference
    result = client.infer(image_url, model_id=model_id)

    # Print results
    print("Inference successful!")
    print(f"Image dimensions: {result.image.width}x{result.image.height}")
    if result.predictions:
        print(f"Found {len(result.predictions)} predictions:")
        for pred in result.predictions:
            print(f"  Class: {pred.class_name}, Confidence: {pred.confidence:.2f}, "
                  f"Box: x={pred.x:.1f}, y={pred.y:.1f}, w={pred.width:.1f}, h={pred.height:.1f}")
    else:
        print("No predictions found.")

except Exception as e:
    print(f"An error occurred during inference: {e}")
    print("Please ensure your ROBOFLOW_API_KEY is correct and the model ID is valid.")