{"id":10288,"library":"tensorrt-cu13-bindings","title":"TensorRT CUDA 13 Python Bindings","description":"tensorrt-cu13-bindings provides Python bindings for NVIDIA's TensorRT, a high-performance deep learning inference library. It enables developers to optimize, validate, and deploy trained deep learning models on NVIDIA GPUs. The library is actively maintained with frequent minor releases, typically on a monthly to bi-monthly cadence, aligned with new TensorRT versions and CUDA compatibility updates.","status":"active","version":"10.16.1.11","language":"en","source_language":"en","source_url":"https://github.com/nvidia/tensorrt","tags":["deep-learning","inference","gpu","nvidia","optimization","cuda","machine-learning"],"install":[{"cmd":"pip install tensorrt-cu13-bindings","lang":"bash","label":"Install for CUDA 13.x"}],"dependencies":[{"reason":"TensorRT is a GPU acceleration library.","package":"NVIDIA GPU","optional":false},{"reason":"TensorRT relies on CUDA for GPU acceleration. The `cu13` in the package name specifies CUDA 13 compatibility.","package":"NVIDIA CUDA Toolkit (13.x recommended)","optional":false},{"reason":"Many TensorRT operations leverage cuDNN for optimized deep learning primitives.","package":"NVIDIA cuDNN","optional":false}],"imports":[{"note":"Standard convention for TensorRT is to import it as 'trt'.","wrong":"import trt","symbol":"tensorrt","correct":"import tensorrt as trt"}],"quickstart":{"code":"import tensorrt as trt\nimport numpy as np\n\n# A simple example: create a dummy network and engine\nTRT_LOGGER = trt.Logger(trt.Logger.WARNING)\n\ndef build_engine():\n    builder = trt.Builder(TRT_LOGGER)\n    network = builder.create_network(1 << int(trt.NetworkDefinitionCreationFlag.EXPLICIT_BATCH))\n    config = builder.create_builder_config()\n\n    # Define input tensor\n    input_tensor = network.add_input(name='input_data', dtype=trt.float32, shape=(1, 3, 224, 224))\n\n    # Define a simple operation (e.g., identity for demonstration)\n    output_tensor = input_tensor\n\n    # Mark output\n    network.mark_output(output_tensor)\n\n    # Build engine (requires GPU and sufficient memory)\n    print(\"Building TensorRT engine...\")\n    serialized_engine = builder.build_serialized_network(network, config)\n    if serialized_engine is None:\n        raise RuntimeError(\"Failed to build TensorRT engine.\")\n    \n    print(\"Engine built successfully.\")\n    return serialized_engine\n\nif __name__ == '__main__':\n    try:\n        serialized_engine = build_engine()\n        runtime = trt.Runtime(TRT_LOGGER)\n        engine = runtime.deserialize_cuda_engine(serialized_engine)\n        \n        print(f\"Engine name: {engine.name}\")\n        print(f\"Number of bindings: {engine.num_bindings}\")\n    except Exception as e:\n        print(f\"An error occurred: {e}\")\n        print(\"Ensure you have a compatible NVIDIA GPU, CUDA Toolkit, and cuDNN installed.\")\n","lang":"python","description":"This quickstart demonstrates how to initialize the TensorRT logger, create a builder and network, define a simple input and output, and build a serialized engine. This process is fundamental for converting a deep learning model into a TensorRT optimized engine. Note that actual inference would require creating an `IExecutionContext` and managing device memory."},"warnings":[{"fix":"Ensure your system's CUDA Toolkit and Python environment meet the requirements for the specific `tensorrt-cuXX-bindings` package you install. The `cuXX` suffix indicates the compatible CUDA major version.","message":"TensorRT regularly updates its required CUDA and Python versions. Version 10.13.2 dropped support for CUDA 11.x and Python < 3.10 for samples/demos, and v10.16 defaulted to CUDA 13.2. Always check the release notes for your specific package version.","severity":"breaking","affected_versions":">=10.13.2"},{"fix":"To access TensorRT samples, clone or download them directly from the official NVIDIA/TensorRT GitHub repository (github.com/nvidia/tensorrt).","message":"As of TensorRT 10.14, sample code is no longer included with the PyPI packages. They are now exclusively hosted in the TensorRT GitHub repository.","severity":"breaking","affected_versions":">=10.14"},{"fix":"If developing custom plugins or using older models, ensure your plugins are updated to implement `IPluginV3` or compatible versions to avoid issues with future TensorRT releases. Refer to the TensorRT Plugin Developer Guide.","message":"TensorRT has been gradually migrating plugins from `IPluginV2`-descendent versions to `IPluginV3`. Older plugin versions are deprecated and will be removed in future releases.","severity":"deprecated","affected_versions":">=10.11"},{"fix":"Verify your system's CUDA Toolkit version and NVIDIA driver compatibility before installing. Install the `tensorrt-cuXX-bindings` package that matches your installed CUDA version. E.g., for CUDA 12, use `tensorrt-cu12-bindings`.","message":"The `tensorrt-cuXX-bindings` package name explicitly ties it to a major CUDA version (e.g., `cu13` for CUDA 13.x). Installing a package that doesn't match your system's CUDA installation (or driver compatibility) will lead to runtime errors.","severity":"gotcha","affected_versions":"All versions"}],"env_vars":null,"last_verified":"2026-04-17T00:00:00.000Z","next_check":"2026-07-16T00:00:00.000Z","problems":[{"fix":"Ensure the NVIDIA CUDA Toolkit and cuDNN are correctly installed and their library paths (e.g., `/usr/local/cuda/lib64`) are included in your system's `LD_LIBRARY_PATH` environment variable. Also verify the installed `tensorrt-cuXX-bindings` package matches your CUDA version.","cause":"The TensorRT core libraries (e.g., `nvinfer.so`, `libnvinfer.so.X`) cannot be found or loaded by the Python bindings. This often indicates missing or incompatible CUDA/cuDNN installations, or an incorrect `LD_LIBRARY_PATH` (on Linux).","error":"tensorrt.tensorrt.UnsatisfiedLinkError: Could not load library: nvinfer.so"},{"fix":"Review your model's operations for TensorRT compatibility, ensure all network inputs and outputs are correctly defined and marked. Increase workspace size if memory is an issue using `config.set_memory_pool_limit(trt.MemoryPoolType.WORKSPACE, ...)`.","cause":"A generic error during the engine building process, often indicating an issue with the network definition (e.g., unsupported layer, invalid input shapes, not marking outputs) or resource constraints.","error":"[TRT] ERROR: ../builder/Network.cpp:xxx Nvinfer error: ..."},{"fix":"Update your NVIDIA GPU drivers to a version compatible with your CUDA Toolkit and the `tensorrt-cuXX-bindings` package. Refer to NVIDIA's CUDA Toolkit release notes for driver compatibility tables.","cause":"The installed NVIDIA driver on your system is too old or incompatible with the CUDA runtime version that the TensorRT package expects. The `tensorrt-cuXX-bindings` package depends on a specific CUDA runtime, which in turn needs a compatible driver.","error":"CUDA driver version is insufficient for CUDA runtime version"},{"fix":"Before building the serialized network, ensure you have called `network.mark_output()` for every tensor you intend to be an output of your TensorRT engine.","cause":"When building an `INetworkDefinition` in TensorRT, you must explicitly mark at least one tensor as an output using `network.mark_output(tensor)`.","error":"[TensorRT] ERROR: Network must have at least one output"}]}