{"id":4576,"library":"inference-gpu","title":"Roboflow Inference (GPU)","description":"Roboflow Inference provides a robust framework for deploying computer vision models across various devices and environments without requiring deep machine learning expertise. This GPU-specific variant leverages CUDA for accelerated inference. Currently at version 1.2.2, the library maintains an active release cadence with frequent patches and minor updates.","status":"active","version":"1.2.2","language":"en","source_language":"en","source_url":"https://github.com/roboflow/inference","tags":["computer-vision","machine-learning","deep-learning","gpu","inference","roboflow"],"install":[{"cmd":"pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu118\npip install inference-gpu","lang":"bash","label":"Install for CUDA 11.8 (adjust CUDA version as needed)"},{"cmd":"pip install inference-gpu","lang":"bash","label":"Install without pre-installing PyTorch (CPU-only or existing PyTorch)"}],"dependencies":[{"reason":"Essential for GPU acceleration; must be installed separately with correct CUDA version prior to inference-gpu for optimal performance.","package":"torch","optional":false},{"reason":"Essential for GPU acceleration; must be installed separately with correct CUDA version prior to inference-gpu for optimal performance.","package":"torchvision","optional":false}],"imports":[{"symbol":"get_roboflow_model","correct":"from inference import get_roboflow_model"},{"note":"Used for real-time video stream processing.","symbol":"Stream","correct":"from inference.core.interfaces.stream.Stream"},{"note":"Path changed in recent 1.x versions. The fully qualified path is now required.","wrong":"from inference import InferenceHTTPClient","symbol":"InferenceHTTPClient","correct":"from inference.core.roboflow_inference import InferenceHTTPClient"}],"quickstart":{"code":"import os\nfrom inference import get_roboflow_model\n\n# Set your Roboflow API key, project ID, and version\nROBOFLOW_API_KEY = os.environ.get('ROBOFLOW_API_KEY', 'YOUR_API_KEY')\nROBOFLOW_MODEL_ID = os.environ.get('ROBOFLOW_MODEL_ID', 'YOUR_PROJECT/YOUR_VERSION')\n\n# Ensure API key is set\nif ROBOFLOW_API_KEY == 'YOUR_API_KEY':\n    print(\"Warning: ROBOFLOW_API_KEY not set. Using placeholder.\")\n\n# Load the model\nprint(f\"Loading model: {ROBOFLOW_MODEL_ID}\")\nmodel = get_roboflow_model(model_id=ROBOFLOW_MODEL_ID, api_key=ROBOFLOW_API_KEY)\n\n# Example inference (using a dummy image path)\n# Replace 'your_image.jpg' with a real image path or PIL Image object\n# For local testing, you might need a dummy image or to adjust the source.\ntry:\n    # This part assumes you have an image file named 'test_image.jpg'\n    # For a real example, replace with a valid image path.\n    # Download a sample image for testing, e.g., from Roboflow Universe.\n    # Example: 'https://i.imgur.com/your_image.jpg'\n    results = model.infer(image_path=\"test_image.jpg\")\n    print(\"Inference successful!\")\n    # print(results)\nexcept FileNotFoundError:\n    print(\"Error: test_image.jpg not found. Please provide a valid image path for inference.\")\nexcept Exception as e:\n    print(f\"An error occurred during inference: {e}\")\n\n# You can also infer on a PIL Image or numpy array\n# from PIL import Image\n# import numpy as np\n# dummy_image = Image.new('RGB', (640, 480), color = 'red')\n# results = model.infer(image=np.array(dummy_image))\n# print(\"Inference successful with dummy image!\")","lang":"python","description":"This quickstart demonstrates how to load a Roboflow model using `get_roboflow_model` and perform inference on an image. It highlights the use of environment variables for API keys and model identification, which is recommended for security and flexibility. Replace 'YOUR_API_KEY' and 'YOUR_PROJECT/YOUR_VERSION' with your actual credentials and model details."},"warnings":[{"fix":"Review model loading and inference logic. If you encounter issues, consult the official documentation for the `inference-models` engine or explicitly configure the legacy backend if necessary (though migrating to the new backend is recommended).","message":"Starting with v1.2.0, `inference-models` became the default backend engine. While the old backend is available in opt-out mode, this change might affect performance or behavior for existing codebases not explicitly configured.","severity":"breaking","affected_versions":">=1.2.0"},{"fix":"Always follow the recommended installation command: `pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cuXXX` before `pip install inference-gpu`, replacing `XXX` with your CUDA version.","message":"For GPU acceleration, `torch` and `torchvision` must be installed *separately* and *prior* to `inference-gpu`, ensuring the correct CUDA version is specified (e.g., `cu118` or `cu121`). Failing to do so will result in CPU-only inference or installation failures.","severity":"gotcha","affected_versions":">=1.0.0"},{"fix":"Upgrade your Python environment to 3.10, 3.11, or 3.12 (as specified by `requires_python`).","message":"Python 3.9 reached End of Life and support for it has been deprecated with `inference` v1.1.0.","severity":"deprecated","affected_versions":">=1.1.0"},{"fix":"Ensure `ROBOFLOW_API_KEY` and `ROBOFLOW_MODEL_ID` (project/version) are correctly set, preferably via environment variables, or passed directly to the model loading function.","message":"API Keys, project IDs, and model versions are mandatory for `get_roboflow_model` and `InferenceHTTPClient`. Incorrect or missing credentials will prevent model loading or inference.","severity":"gotcha","affected_versions":"All versions"}],"env_vars":null,"last_verified":"2026-04-12T00:00:00.000Z","next_check":"2026-07-11T00:00:00.000Z"}