{"id":2619,"library":"nvidia-cudnn-frontend","title":"NVIDIA cuDNN Frontend","description":"The `nvidia-cudnn-frontend` is a Python library that provides a high-level, user-friendly API to interact with the cuDNN deep learning library backend. It facilitates the creation and execution of optimized tensor operations, including various fusions and custom kernels, specifically designed for NVIDIA GPUs. It is currently at version 1.22.1 and maintains an active release cadence, often aligning with new cuDNN backend releases.","status":"active","version":"1.22.1","language":"en","source_language":"en","source_url":"https://github.com/nvidia/cudnn-frontend","tags":["AI/ML","GPU","NVIDIA","cuDNN","Deep Learning","Performance","CUDA","backend"],"install":[{"cmd":"pip install nvidia-cudnn-frontend","lang":"bash","label":"Install latest version"}],"dependencies":[{"reason":"Commonly used for tensor operations and integration with PyTorch models, as many custom ops are PyTorch-compatible.","package":"torch","optional":true}],"imports":[{"symbol":"cudnn_frontend","correct":"import cudnn_frontend"}],"quickstart":{"code":"import cudnn_frontend\nimport torch\n\n# Ensure CUDA device is available\nif not torch.cuda.is_available():\n    raise RuntimeError(\"CUDA is not available. This library requires a CUDA-enabled GPU.\")\n\n# Example: Create and execute a simple convolution graph\n# Define input and weight tensors on CUDA\nx = torch.randn(1, 1, 28, 28, device=\"cuda\", dtype=torch.float32)\nw = torch.randn(16, 1, 3, 3, device=\"cuda\", dtype=torch.float32)\n\n# Create a cuDNN frontend graph\ngraph = cudnn_frontend.create_graph({\"fp8_mode\": False}) # fp8_mode can be set to True for FP8 operations\n\n# Make input tensors for the graph from PyTorch tensors\nX = graph.make_input_tensor(\n    \"X\", cudnn_frontend.DataType.FLOAT, x.shape, x.stride()\n)\nW = graph.make_input_tensor(\n    \"W\", cudnn_frontend.DataType.FLOAT, w.shape, w.stride()\n)\n\n# Define a convolution operation\nY = graph.make_convolution(\n    X, W, padding=[1, 1], stride=[1, 1], dilation=[1, 1]\n)\n# Mark the output tensor\nY.set_output()\n\n# Build the operation graph, create execution plans, and check support\ngraph.build_operation_graph()\ngraph.create_execution_plans(\n    [cudnn_frontend.heur_mode.A, cudnn_frontend.heur_mode.FALLBACK]\n)\ngraph.check_support()\ngraph.build_plans()\n\n# Allocate the output tensor on CUDA\ny_out = torch.empty(Y.get_output_tensors()[0].get_dim(), device=\"cuda\", dtype=torch.float32)\n\n# Execute the graph\ngraph.execute([x, w], [y_out])\n\nprint(\"Graph execution successful!\")\nprint(f\"Output tensor shape: {y_out.shape}\")","lang":"python","description":"This quickstart demonstrates how to initialize cuDNN frontend, define input tensors, create a convolution operation within a graph, build and execute the graph, and retrieve the output using PyTorch tensors on a CUDA-enabled GPU."},"warnings":[{"fix":"Ensure that the NVIDIA CUDA Toolkit and a supported version of the cuDNN backend library are correctly installed and configured on your system (e.g., `LD_LIBRARY_PATH` or system paths). Consult the official cuDNN documentation for installation instructions and compatibility matrix.","message":"The `nvidia-cudnn-frontend` Python package requires a compatible installation of the NVIDIA CUDA Toolkit and the cuDNN native C++ library on the system. Installing the Python package alone is insufficient for functionality.","severity":"breaking","affected_versions":"All versions"},{"fix":"Always refer to the release notes of your `nvidia-cudnn-frontend` version to identify the recommended `cuDNN` backend and `CUDA Toolkit` versions. For example, v1.22.1 is recommended for cuDNN 9.20.0 and later. Ensure your system setup matches these recommendations.","message":"Specific `nvidia-cudnn-frontend` versions are often recommended for particular `cuDNN` backend and `CUDA Toolkit` versions. Mismatched versions can lead to runtime errors, performance issues, or inability to leverage new features.","severity":"gotcha","affected_versions":"All versions"},{"fix":"Review your code for any dependencies on internal `v0.x` API elements of `cudnn_frontend`. Update your code to use the officially exposed public API functions and methods, which now directly map to the backend calls.","message":"In version 1.18.0, the library internally transitioned away from using the older `v0.x API` and now directly calls the cuDNN backend API. This change might break compatibility for users who were relying on or interacting with internal `v0.x` API constructs.","severity":"breaking","affected_versions":">=1.18.0"},{"fix":"If encountering issues, ensure you are using `nvidia-cudnn-frontend` v1.19.1 or later. If you manage `pybind11` versions explicitly, ensure compatibility or allow `pip` to manage it for `nvidia-cudnn-frontend`.","message":"Version 1.19.1 was released to address issues with `pybind11` versions and restore `cuda-12` toolkit support accidentally dropped in 1.19.0. Older or incompatible `pybind11` installations can cause installation or runtime failures.","severity":"gotcha","affected_versions":">=1.19.0"}],"env_vars":null,"last_verified":"2026-04-10T00:00:00.000Z","next_check":"2026-07-09T00:00:00.000Z"}