TensorRT Libraries (CUDA 12.x)

10.16.1.11 verified Mon Apr 27 auth: no python

NVIDIA TensorRT libraries for CUDA 12.x, providing core runtime and plugin libraries for high-performance deep learning inference. Current version 10.16.1.11. Released on PyPI as a separate wheel to isolate CUDA 12 dependencies; updates follow TensorRT releases (approximately quarterly).

pip install tensorrt-cu12-libs==10.16.1.11

Common errors

error ModuleNotFoundError: No module named 'tensorrt' ↓

cause Python bindings not installed; only the libs package is installed.

fix

Install tensorrt (full) or tensorrt-cu12-bindings: pip install tensorrt-cu12-libs==10.16.1.11 tensorrt-cu12-bindings==10.16.1.11

error ImportError: libnvinfer.so.10: cannot open shared object file: No such file or directory ↓

cause LD_LIBRARY_PATH not set or missing CUDA libraries.

fix

Add TensorRT lib directory to LD_LIBRARY_PATH, e.g., export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/tensorrt/lib (or the site-packages tensorrt directory).

error tensorrt.RuntimeError: [runtime.cpp::validateEngine::625] Error Code 1: Serialization (Serialized engine version X does not match TensorRT version Y) ↓

cause Engine was built with a different TensorRT version than the runtime.

fix

Rebuild engine with the same TensorRT version as the runtime, or use a compatible version.

error AssertionError: (shape) vs (expected shape) - TensorRT: The engine plan file is not compatible with this version of TensorRT. ↓

cause Attempting to use an engine built with a different TensorRT major version.

fix

Rebuild the engine with the exact TensorRT version in use.

Warnings

breaking The tensorrt-cu12-libs package only supports CUDA 12.x. On CUDA 11 systems, use tensorrt-cu11-libs instead. Mixing CUDA versions causes cryptic runtime errors. ↓

fix Check CUDA version via `nvcc --version` and install matching -cuXX- variant.

breaking All tensorrt-* packages (libs, bindings, python) must be the same version. Version mismatch leads to import errors or segmentation faults. ↓

fix Install exact same version for all components, e.g., pip install tensorrt-cu12-libs==10.16.1.11 tensorrt-cu12-bindings==10.16.1.11 tensorrt==10.16.1.11

gotcha The tensorrt-cu12-libs wheel does not include the Python bindings; you must install tensorrt or tensorrt-cu12-bindings separately. ↓

fix Also install tensorrt (full) or tensorrt-cu12-bindings (Python bindings only).

deprecated TensorRT 10.16 defaults to CUDA 13.2; the cu12 variant is available for legacy CUDA 12 compatibility but may be dropped in future releases. ↓

fix Plan migration to CUDA 13.2 to use the default tensorrt package.

Imports

tensorrt
wrong
```
from tensorrt import *
```
correct
```
import tensorrt as trt
```
Importing all with * pollutes namespace; use alias trt.

Quickstart

Verifies TensorRT installation and creates a trivial identity engine.

import tensorrt as trt
import os

# Verify installation
print(f"TensorRT version: {trt.__version__}")

# Create a logger
logger = trt.Logger(trt.Logger.WARNING)

# Build a simple identity network
builder = trt.Builder(logger)
network = builder.create_network(1 << int(trt.NetworkDefinitionCreationFlag.EXPLICIT_BATCH))

# Add an input tensor
input_tensor = network.add_input(name="input", dtype=trt.float32, shape=(1, 3, 224, 224))

# Add an identity layer (pass-through)
identity = network.add_identity(input_tensor)

# Mark output
network.mark_output(identity.get_output(0))

# Build engine
config = builder.create_builder_config()
config.set_memory_pool_limit(trt.MemoryPoolType.WORKSPACE, 1 << 30)  # 1 GB
engine_bytes = builder.build_serialized_network(network, config)

if engine_bytes:
    print("Engine built successfully")
else:
    print("Engine build failed")