OpenVINO Development Tools

2024.6.0 · active · verified Thu Apr 16

OpenVINO™ Development Tools (openvino-dev) is Intel's comprehensive toolkit for optimizing and deploying AI models for inference across various Intel hardware. It includes the OpenVINO™ Runtime, Model Optimizer, and Post-Training Optimization Tool. The current version is 2024.6.0. Intel typically releases major updates quarterly, providing a consistent cadence of new features and performance improvements.

Common errors

```
RuntimeError: Cannot find plugin to use for device <device_name>
```
cause OpenVINO cannot find the necessary plugin (driver) to run inference on the specified device. This often happens with 'GPU' if Intel graphics drivers or OpenCL/oneAPI runtimes are not installed, or with 'NPU' if the corresponding hardware and drivers are missing.

fix Verify that the target device is physically present and its respective drivers/run-time components are correctly installed. Use `ov.Core().available_devices` to check which devices are recognized by OpenVINO. Fallback to 'CPU' if other devices are unavailable.
```
AttributeError: 'openvino.runtime.Core' object has no attribute 'read_network'
```
cause This error indicates usage of the old API (`read_network` was part of `IECore`) with the new `openvino.runtime.Core` object. The `read_network` method was deprecated and removed.

fix Replace `core.read_network(...)` with `core.read_model(...)`. The new API uses `read_model` to load an IR or other framework model.
```
RuntimeError: Check 'device != ""' failed at src/inference_engine/ie_core.cpp:...
```
cause An empty string was passed as the device name to `core.compile_model()` or `core.get_versions()`, which is not a valid device identifier.

fix Always provide a valid device string like 'CPU', 'GPU', 'NPU', or 'AUTO'. Ensure any environment variables or configuration values used for the device string are not empty.

Warnings

breaking Major API changes occurred in OpenVINO 2023.0+ concerning `IECore` to `Core`, `read_network` to `read_model`, and `load_network` to `compile_model`.
Fix: Update your code to use `openvino.runtime.Core`, `core.read_model(...)`, and `core.compile_model(...)`. For example, `core = ov.Core()` instead of `core = ov.inference_engine.IECore()`.
breaking The Model Optimizer functionality, previously accessed via `openvino.tools.mo.convert_model`, is now directly exposed as `openvino.convert_model`.
Fix: Change import statements and calls from `from openvino.tools.mo import convert_model` to `from openvino import convert_model`.
gotcha Device selection (e.g., 'CPU', 'GPU', 'NPU') requires the correct drivers and hardware. Using 'AUTO' or a non-existent device may lead to runtime errors or fallback to CPU.
Fix: Ensure the target device is present and drivers are installed. Use `core.available_devices` to list supported devices. For maximum compatibility, use 'CPU' or 'AUTO'.

Install

pip install openvino-dev Install OpenVINO Development Tools

Imports

Core
wrong:
```
from openvino.inference_engine import IECore
```
correct:
```
from openvino.runtime import Core
```
The `IECore` class was deprecated and removed in OpenVINO 2023.0+, replaced by `openvino.runtime.Core`.
Model
```
from openvino.runtime import Model
```
Represents the OpenVINO model graph.
CompiledModel
```
from openvino.runtime import CompiledModel
```
Represents a model compiled for a specific device.
convert_model
wrong:
```
from openvino.tools.mo import convert_model
```
correct:
```
from openvino import convert_model
```
The recommended path for `convert_model` (formerly Model Optimizer) is now directly under `openvino`.
Type
```
from openvino.runtime import Type
```
Used for defining tensor data types in programmatic model creation.
opset12
```
from openvino.runtime import opset12
```
Provides OpenVINO operations for programmatic model creation (e.g., opset12.parameter, opset12.add).

Quickstart

This quickstart demonstrates how to create an OpenVINO Core object, define a simple model programmatically, compile it for a target device, and perform inference. It uses the `openvino.runtime` API introduced in OpenVINO 2023.0+ and avoids external model files or deep learning frameworks for simplicity.

import openvino as ov
import numpy as np
import os

# 1. Create a Core object
core = ov.Core()

# 2. Define a simple model programmatically (e.g., a single addition operation)
# This avoids external files and frameworks for a minimal example.
input_a = ov.opset12.parameter([1, 3, 224, 224], ov.Type.f32, name="input_a")
input_b = ov.opset12.parameter([1, 3, 224, 224], ov.Type.f32, name="input_b")
result_op = ov.opset12.add(input_a, input_b, name="add_result")
model = ov.Model([result_op], [input_a, input_b], "simple_add_model")

# 3. Compile the model for a specific device (e.g., 'CPU', 'GPU', 'NPU')
# Use 'AUTO' to let OpenVINO select the best available device.
device = os.environ.get("OPENVINO_DEVICE", "CPU") # Default to CPU
try:
    compiled_model = core.compile_model(model, device)
    print(f"Model compiled successfully for {device} device.")
except RuntimeError as e:
    print(f"Warning: Could not compile model for device '{device}': {e}")
    print("Falling back to CPU if available.")
    device = "CPU"
    compiled_model = core.compile_model(model, device)

# 4. Prepare input data
input_data_a = np.random.rand(1, 3, 224, 224).astype(np.float32)
input_data_b = np.random.rand(1, 3, 224, 224).astype(np.float32)

# 5. Perform inference
# Inputs can be passed as a list, dictionary, or single tensor depending on model
outputs = compiled_model([input_data_a, input_data_b])

# 6. Process results
print(f"Inference successful on {device} device.")
# The output is a list of numpy arrays, one for each output of the model
print(f"Output shape: {outputs[0].shape}, dtype: {outputs[0].dtype}")

view raw JSON →