NVIDIA CUDA NVRTC for CUDA 12.x
raw JSON → 12.9.86 verified Sat Apr 11 auth: no python quickstart: draft
NVRTC (NVIDIA Runtime Compiler) is a library designed for runtime compilation of CUDA C++ source code into PTX (Parallel Thread Execution) assembly. This Python package, `nvidia-cuda-nvrtc-cu12`, provides the native runtime libraries (DLLs/SOs) for CUDA 12.x, enabling dynamic code generation and execution on NVIDIA GPUs. It's a fundamental component of the CUDA Toolkit, actively maintained by NVIDIA, and crucial for other Python libraries that leverage JIT CUDA compilation.
pip install nvidia-cuda-nvrtc-cu12 Common errors
error nvrtc: error: failed to open nvrtc-builtins64_124.dll. Make sure that nvrtc-builtins64_124.dll is installed correctly. ↓
cause This error typically occurs on Windows when the NVRTC built-in library (or a similar NVRTC DLL/shared object file like `libnvrtc.so` on Linux) cannot be found or loaded by the application at runtime, often due to incorrect installation, PATH issues, or environment mismatches.
fix
Ensure that the CUDA Toolkit is correctly installed and that the directory containing
nvrtc-builtins64_XXX.dll (e.g., C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.x\bin on Windows) or libnvrtc.so (e.g., /usr/local/cuda/lib64 on Linux) is included in your system's PATH or LD_LIBRARY_PATH environment variable. If using a Python environment, activate it and ensure necessary CUDA libraries are discoverable, or consider reinstalling the nvidia-cuda-nvrtc-cu12 package within that environment. error RuntimeError: Could not find libnvrtc.so. Please make sure CUDA is installed. ↓
cause This error indicates that a Python library attempting to use NVRTC cannot locate the `libnvrtc.so` shared library on Linux-based systems, usually because the CUDA Toolkit is not installed, or its library paths are not correctly configured for the Python environment.
fix
Verify that the CUDA Toolkit is installed and the
libnvrtc.so file exists in a standard CUDA library directory (e.g., /usr/local/cuda/lib64). Ensure that this directory is included in your LD_LIBRARY_PATH environment variable. If using an Anaconda/Conda environment, ensure CUDA-related packages are installed correctly within that environment, or try setting CUDA_HOME to your CUDA installation path. error NVRTCError: NVRTC_ERROR_COMPILATION (6) ↓
cause This error signifies a failure during the runtime compilation of CUDA C++ source code by NVRTC, often due to syntax errors in the provided CUDA kernel code, incompatible compiler options, or an issue with the NVRTC environment itself.
fix
Review the CUDA C++ source code being passed to NVRTC for any syntax errors or unsupported features. Check the compilation options being used (e.g.,
--std=c++11). Ensure that the CUDA Toolkit and NVRTC library versions are compatible with the code and the consuming application. Sometimes, updating or downgrading dependent libraries like CuPy might resolve underlying compatibility issues. error ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts. torch 2.3.1 requires nvidia-cuda-nvrtc-cu12==12.1.105; ... but you have nvidia-cuda-nvrtc-cu12 12.3.107 which is incompatible. ↓
cause This pip error occurs when there's a version mismatch between `nvidia-cuda-nvrtc-cu12` (or other `nvidia-cuda-*` packages) and other core deep learning libraries like PyTorch or TensorFlow, which have strict dependencies on specific CUDA component versions.
fix
To resolve dependency conflicts, install PyTorch (or TensorFlow) and its CUDA dependencies together using the official installation instructions (e.g., from
pytorch.org) which provide a command to install all compatible nvidia-cuda-* packages. Alternatively, manually specify compatible versions for nvidia-cuda-nvrtc-cu12 and other nvidia-cuda-* packages that match the requirements of your main deep learning framework. Using a fresh virtual environment is highly recommended. Warnings
breaking CUDA Python wheels, including `nvidia-cuda-nvrtc-cu12`, require a compatible NVIDIA GPU driver installed on the system. Mismatched driver versions (e.g., an older driver with a newer CUDA toolkit version or vice-versa) can lead to runtime errors or GPU features being unavailable. ↓
fix Ensure your NVIDIA GPU driver is up-to-date and compatible with the CUDA toolkit version (indicated by 'cu12' in the package name). Consult NVIDIA's CUDA compatibility matrix.
gotcha This package is intended for 'runtime use' and provides native shared libraries (like `.dll` or `.so` files). It *does not* include developer tools, headers, or the `nvcc` compiler itself. For full CUDA development workflows, the complete NVIDIA CUDA Toolkit must be installed separately. ↓
fix If you need to compile CUDA C++ code or access CUDA headers, install the full CUDA Toolkit from NVIDIA's developer website alongside the Python runtime components.
gotcha The `nvidia-cuda-nvrtc-cu12` package does not expose a direct Python API or symbols for import by end-user Python code. Its primary role is to provide the native NVRTC libraries that other Python packages (e.g., PyTorch, CuPy, JAX) link against and use internally for Just-In-Time (JIT) compilation of CUDA C++ kernels. ↓
fix Do not expect to `import nvrtc` or similar directly. Instead, utilize higher-level Python libraries that wrap CUDA functionality. This package is an underlying runtime dependency for them.
gotcha Security analyses of `nvidia-cuda-nvrtc-cu12` have occasionally flagged 'reduced effectiveness mitigations' for vulnerabilities when statically linked libraries from different toolchain versions are combined. This can impact the integrity of control flow protection mechanisms. ↓
fix For applications with high security requirements, regularly review security reports (e.g., from tools like ReversingLabs) and ensure consistent toolchain versions for all linked components. Consult programming language toolchain documentation (e.g., for Microsoft VisualStudio, use `/guard:cf`).
gotcha To fully test or utilize `nvidia-cuda-nvrtc-cu12` effectively, ensure higher-level CUDA-enabled libraries like PyTorch or CuPy are installed. As this package primarily provides underlying runtime functionality, direct verification of CUDA availability or JIT compilation might be limited without these consumers. ↓
fix Install PyTorch, CuPy, or other Python libraries that utilize `nvidia-cuda-nvrtc-cu12` to enable comprehensive testing and full functional access.
breaking This package is a placeholder on PyPI.org. To install it, you must first configure pip to use the NVIDIA Python Package Index. ↓
fix Install the `nvidia-pyindex` package first to configure pip to use the NVIDIA Python Package Index, then install `nvidia-cuda-nvrtc-cu12`. Example: `pip install nvidia-pyindex && pip install nvidia-cuda-nvrtc-cu12`
Quickstart draft last tested: 2026-04-24
import os
# This package primarily provides native runtime libraries (DLLs/SOs)
# for CUDA's NVRTC component, which are consumed by other higher-level
# CUDA-enabled Python libraries (e.g., PyTorch, CuPy) for JIT compilation.
# There is no direct Python API exposed by this specific package for end-user import.
# The following code demonstrates verifying CUDA's availability, which implicitly
# relies on the correctly installed underlying CUDA runtime components like NVRTC.
try:
import torch
print(f"PyTorch CUDA available: {torch.cuda.is_available()}")
if torch.cuda.is_available():
print(f"PyTorch CUDA version: {torch.version.cuda}")
print(f"Current CUDA device: {torch.cuda.get_device_name(0)}")
except ImportError:
print("PyTorch not installed. Cannot verify CUDA availability via PyTorch.")
try:
import cupy
print(f"CuPy CUDA available: {cupy.cuda.is_available()}")
if cupy.cuda.is_available():
print(f"CuPy CUDA version: {cupy.cuda.runtime.get_version()}")
print(f"Current CUDA device: {cupy.cuda.Device(0).name}")
except ImportError:
print("CuPy not installed. Cannot verify CUDA availability via CuPy.")
# A successful installation of nvidia-cuda-nvrtc-cu12 means these libraries
# are available for use by such frameworks.