TensorRT CUDA 13 Python Bindings

Common errors

• tensorrt.tensorrt.UnsatisfiedLinkError: Could not load library: nvinfer.so

  cause The TensorRT core libraries (e.g., `nvinfer.so`, `libnvinfer.so.X`) cannot be found or loaded by the Python bindings. This often indicates missing or incompatible CUDA/cuDNN installations, or an incorrect `LD_LIBRARY_PATH` (on Linux).
  fix Ensure the NVIDIA CUDA Toolkit and cuDNN are correctly installed and their library paths (e.g., `/usr/local/cuda/lib64`) are included in your system's `LD_LIBRARY_PATH` environment variable. Also verify the installed `tensorrt-cuXX-bindings` package matches your CUDA version.

• [TRT] ERROR: ../builder/Network.cpp:xxx Nvinfer error: ...

  cause A generic error during the engine building process, often indicating an issue with the network definition (e.g., unsupported layer, invalid input shapes, not marking outputs) or resource constraints.
  fix Review your model's operations for TensorRT compatibility, ensure all network inputs and outputs are correctly defined and marked. Increase workspace size if memory is an issue using `config.set_memory_pool_limit(trt.MemoryPoolType.WORKSPACE, ...)`.

• CUDA driver version is insufficient for CUDA runtime version

  cause The installed NVIDIA driver on your system is too old or incompatible with the CUDA runtime version that the TensorRT package expects. The `tensorrt-cuXX-bindings` package depends on a specific CUDA runtime, which in turn needs a compatible driver.
  fix Update your NVIDIA GPU drivers to a version compatible with your CUDA Toolkit and the `tensorrt-cuXX-bindings` package. Refer to NVIDIA's CUDA Toolkit release notes for driver compatibility tables.

• [TensorRT] ERROR: Network must have at least one output

  cause When building an `INetworkDefinition` in TensorRT, you must explicitly mark at least one tensor as an output using `network.mark_output(tensor)`.
  fix Before building the serialized network, ensure you have called `network.mark_output()` for every tensor you intend to be an output of your TensorRT engine.

Warnings

• breaking TensorRT regularly updates its required CUDA and Python versions. Version 10.13.2 dropped support for CUDA 11.x and Python < 3.10 for samples/demos, and v10.16 defaulted to CUDA 13.2. Always check the release notes for your specific package version.
  Fix: Ensure your system's CUDA Toolkit and Python environment meet the requirements for the specific `tensorrt-cuXX-bindings` package you install. The `cuXX` suffix indicates the compatible CUDA major version.
• breaking As of TensorRT 10.14, sample code is no longer included with the PyPI packages. They are now exclusively hosted in the TensorRT GitHub repository.
  Fix: To access TensorRT samples, clone or download them directly from the official NVIDIA/TensorRT GitHub repository (github.com/nvidia/tensorrt).
• deprecated TensorRT has been gradually migrating plugins from `IPluginV2`-descendent versions to `IPluginV3`. Older plugin versions are deprecated and will be removed in future releases.
  Fix: If developing custom plugins or using older models, ensure your plugins are updated to implement `IPluginV3` or compatible versions to avoid issues with future TensorRT releases. Refer to the TensorRT Plugin Developer Guide.
• gotcha The `tensorrt-cuXX-bindings` package name explicitly ties it to a major CUDA version (e.g., `cu13` for CUDA 13.x). Installing a package that doesn't match your system's CUDA installation (or driver compatibility) will lead to runtime errors.
  Fix: Verify your system's CUDA Toolkit version and NVIDIA driver compatibility before installing. Install the `tensorrt-cuXX-bindings` package that matches your installed CUDA version. E.g., for CUDA 12, use `tensorrt-cu12-bindings`.

Install

• pip install tensorrt-cu13-bindings Install for CUDA 13.x

Imports

• tensorrt
  wrong:

  import trt

  correct:

  import tensorrt as trt

  Standard convention for TensorRT is to import it as 'trt'.

Quickstart

This quickstart demonstrates how to initialize the TensorRT logger, create a builder and network, define a simple input and output, and build a serialized engine. This process is fundamental for converting a deep learning model into a TensorRT optimized engine. Note that actual inference would require creating an `IExecutionContext` and managing device memory.

import tensorrt as trt
import numpy as np

# A simple example: create a dummy network and engine
TRT_LOGGER = trt.Logger(trt.Logger.WARNING)

def build_engine():
    builder = trt.Builder(TRT_LOGGER)
    network = builder.create_network(1 << int(trt.NetworkDefinitionCreationFlag.EXPLICIT_BATCH))
    config = builder.create_builder_config()

    # Define input tensor
    input_tensor = network.add_input(name='input_data', dtype=trt.float32, shape=(1, 3, 224, 224))

    # Define a simple operation (e.g., identity for demonstration)
    output_tensor = input_tensor

    # Mark output
    network.mark_output(output_tensor)

    # Build engine (requires GPU and sufficient memory)
    print("Building TensorRT engine...")
    serialized_engine = builder.build_serialized_network(network, config)
    if serialized_engine is None:
        raise RuntimeError("Failed to build TensorRT engine.")
    
    print("Engine built successfully.")
    return serialized_engine

if __name__ == '__main__':
    try:
        serialized_engine = build_engine()
        runtime = trt.Runtime(TRT_LOGGER)
        engine = runtime.deserialize_cuda_engine(serialized_engine)
        
        print(f"Engine name: {engine.name}")
        print(f"Number of bindings: {engine.num_bindings}")
    except Exception as e:
        print(f"An error occurred: {e}")
        print("Ensure you have a compatible NVIDIA GPU, CUDA Toolkit, and cuDNN installed.")