Roboflow Inference (GPU)

1.2.2 · active · verified Sun Apr 12

Roboflow Inference provides a robust framework for deploying computer vision models across various devices and environments without requiring deep machine learning expertise. This GPU-specific variant leverages CUDA for accelerated inference. Currently at version 1.2.2, the library maintains an active release cadence with frequent patches and minor updates.

Warnings

breaking Starting with v1.2.0, `inference-models` became the default backend engine. While the old backend is available in opt-out mode, this change might affect performance or behavior for existing codebases not explicitly configured.
Fix: Review model loading and inference logic. If you encounter issues, consult the official documentation for the `inference-models` engine or explicitly configure the legacy backend if necessary (though migrating to the new backend is recommended).
gotcha For GPU acceleration, `torch` and `torchvision` must be installed *separately* and *prior* to `inference-gpu`, ensuring the correct CUDA version is specified (e.g., `cu118` or `cu121`). Failing to do so will result in CPU-only inference or installation failures.
Fix: Always follow the recommended installation command: `pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cuXXX` before `pip install inference-gpu`, replacing `XXX` with your CUDA version.
deprecated Python 3.9 reached End of Life and support for it has been deprecated with `inference` v1.1.0.
Fix: Upgrade your Python environment to 3.10, 3.11, or 3.12 (as specified by `requires_python`).
gotcha API Keys, project IDs, and model versions are mandatory for `get_roboflow_model` and `InferenceHTTPClient`. Incorrect or missing credentials will prevent model loading or inference.
Fix: Ensure `ROBOFLOW_API_KEY` and `ROBOFLOW_MODEL_ID` (project/version) are correctly set, preferably via environment variables, or passed directly to the model loading function.

Install

pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu118 pip install inference-gpu Install for CUDA 11.8 (adjust CUDA version as needed)
pip install inference-gpu Install without pre-installing PyTorch (CPU-only or existing PyTorch)

Imports

get_roboflow_model

from inference import get_roboflow_model

Stream
```
from inference.core.interfaces.stream.Stream
```
Used for real-time video stream processing.
InferenceHTTPClient
```
from inference.core.roboflow_inference import InferenceHTTPClient
```
Path changed in recent 1.x versions. The fully qualified path is now required.

Quickstart

This quickstart demonstrates how to load a Roboflow model using `get_roboflow_model` and perform inference on an image. It highlights the use of environment variables for API keys and model identification, which is recommended for security and flexibility. Replace 'YOUR_API_KEY' and 'YOUR_PROJECT/YOUR_VERSION' with your actual credentials and model details.

import os
from inference import get_roboflow_model

# Set your Roboflow API key, project ID, and version
ROBOFLOW_API_KEY = os.environ.get('ROBOFLOW_API_KEY', 'YOUR_API_KEY')
ROBOFLOW_MODEL_ID = os.environ.get('ROBOFLOW_MODEL_ID', 'YOUR_PROJECT/YOUR_VERSION')

# Ensure API key is set
if ROBOFLOW_API_KEY == 'YOUR_API_KEY':
    print("Warning: ROBOFLOW_API_KEY not set. Using placeholder.")

# Load the model
print(f"Loading model: {ROBOFLOW_MODEL_ID}")
model = get_roboflow_model(model_id=ROBOFLOW_MODEL_ID, api_key=ROBOFLOW_API_KEY)

# Example inference (using a dummy image path)
# Replace 'your_image.jpg' with a real image path or PIL Image object
# For local testing, you might need a dummy image or to adjust the source.
try:
    # This part assumes you have an image file named 'test_image.jpg'
    # For a real example, replace with a valid image path.
    # Download a sample image for testing, e.g., from Roboflow Universe.
    # Example: 'https://i.imgur.com/your_image.jpg'
    results = model.infer(image_path="test_image.jpg")
    print("Inference successful!")
    # print(results)
except FileNotFoundError:
    print("Error: test_image.jpg not found. Please provide a valid image path for inference.")
except Exception as e:
    print(f"An error occurred during inference: {e}")

# You can also infer on a PIL Image or numpy array
# from PIL import Image
# import numpy as np
# dummy_image = Image.new('RGB', (640, 480), color = 'red')
# results = model.infer(image=np.array(dummy_image))
# print("Inference successful with dummy image!")

view raw JSON →