NVIDIA cuDNN Runtime Libraries for CUDA 12

9.20.0.48 · active · verified Sat Mar 28

The `nvidia-cudnn-cu12` package provides the NVIDIA CUDA Deep Neural Network (cuDNN) runtime libraries, which are GPU-accelerated primitives essential for deep neural network operations such as convolutions, attention, and matrix multiplication. It acts as a critical low-level dependency, enabling deep learning frameworks like TensorFlow and PyTorch to efficiently leverage NVIDIA GPUs. This specific package targets CUDA 12.x environments. The current version is 9.20.0.48, with releases frequently updated to align with new CUDA Toolkit versions and cuDNN backend enhancements.

Warnings

breaking Starting with CUDA 12.5 and later, cuDNN is no longer bundled directly within the CUDA Toolkit installer. This change requires users (especially C++ toolchain developers) to manage cuDNN installation and versioning separately, although `pip install nvidia-cudnn-cu12` simplifies this for Python environments.
Fix: Ensure you are installing `nvidia-cudnn-cu12` (or the appropriate CUDA version) via pip, or manually managing separate cuDNN archives if not using Python wheels, and verify compatibility with your CUDA Toolkit version.
gotcha Direct Python API calls for `nvidia-cudnn-cu12` are not available. This package provides the low-level runtime binaries. To programmatically interact with cuDNN functionality in Python (e.g., build computation graphs), you must install the `nvidia-cudnn-frontend` package separately and import it as `cudnn`.
Fix: If you intend to use cuDNN's API directly in Python, install `nvidia-cudnn-frontend` via `pip install nvidia-cudnn-frontend` and then `import cudnn` in your Python code.
gotcha Version compatibility between `nvidia-cudnn-cu12`, the installed NVIDIA CUDA Toolkit, and your deep learning framework (e.g., TensorFlow, PyTorch) is crucial. Frameworks are often built against specific cuDNN versions. Installing a standalone `nvidia-cudnn-cu12` might not be compatible with the version your framework expects, leading to runtime errors (e.g., 'DLL load failed' or 'cuDNN initialization error').
Fix: Always refer to the official documentation of your deep learning framework for recommended or required CUDA and cuDNN versions. When possible, allow the framework's installation (e.g., `pip install tensorflow[and-cuda]`) to manage cuDNN dependencies, or carefully match versions if installing separately.
gotcha `nvidia-cudnn-cu12` implies compatibility with CUDA Toolkit 12.x. Using it with an older or incompatible CUDA Toolkit version installed on your system can lead to runtime issues or failures in GPU acceleration.
Fix: Ensure your installed NVIDIA CUDA Toolkit version is 12.x and that your GPU drivers are up-to-date and compatible with CUDA 12.x.

Install

pip install nvidia-cudnn-cu12 Install latest version
pip install nvidia-cudnn-cu12==9.20.0.48 Install specific version

Imports

cudnn
```
import cudnn
```
The `nvidia-cudnn-cu12` package provides the underlying cuDNN runtime libraries, not a direct Python API for computation. Python interaction for defining and executing operations is typically achieved through the `nvidia-cudnn-frontend` package (installed separately via `pip install nvidia-cudnn-frontend`), which then imports as `cudnn`.

Quickstart

This quickstart demonstrates how to verify that a deep learning framework like TensorFlow can detect and utilize your GPU, and implicitly, the underlying cuDNN runtime. Successful execution of this code confirms that `nvidia-cudnn-cu12` is likely correctly installed and accessible by TensorFlow. Note that the cuDNN version reported by TensorFlow is the version it was *built with*, not necessarily the exact version dynamically loaded, though they should be compatible.

import tensorflow as tf
import os

# Ensure TensorFlow doesn't pre-allocate all GPU memory
os.environ['TF_FORCE_GPU_ALLOW_GROWTH'] = 'true'

# Check if TensorFlow can detect and use GPUs
gpus = tf.config.list_physical_devices('GPU')

if gpus:
    print(f"TensorFlow detected the following GPUs: {gpus}")
    try:
        # Limit GPU memory growth to avoid allocating all memory at once (alternative to env var)
        for gpu in gpus:
            tf.config.experimental.set_memory_growth(gpu, True)
        print("GPU memory growth set to True.")
    except RuntimeError as e:
        # Memory growth must be set before GPUs have been initialized
        print(f"Error setting memory growth: {e}")
    print(f"TensorFlow is built with CUDA: {tf.test.is_built_with_cuda()}")
    # TensorFlow's built-in cuDNN version (indicates what TF was compiled with)
    print(f"TensorFlow's built-in cuDNN version: {tf.sysconfig.get_build_info().get('CUDNN_VERSION', 'N/A')}")

    # A small operation to trigger GPU usage if available
    try:
        with tf.device('/GPU:0'):
            a = tf.constant([[1.0, 2.0], [3.0, 4.0]])
            b = tf.constant([[1.0, 1.0], [1.0, 1.0]])
            c = tf.matmul(a, b)
            print(f"Simple matrix multiplication on GPU: {c.numpy()}")
    except RuntimeError as e:
        print(f"Could not run on GPU: {e}. Running on CPU instead.")
        a = tf.constant([[1.0, 2.0], [3.0, 4.0]])
        b = tf.constant([[1.0, 1.0], [1.0, 1.0]])
        c = tf.matmul(a, b)
        print(f"Simple matrix multiplication on CPU: {c.numpy()}")
else:
    print("TensorFlow did not detect any GPUs. Please ensure CUDA and cuDNN are correctly installed and configured.")

view raw JSON →