Hugging Face Kernels

0.13.0 · active · verified Mon Apr 13

The `kernels` Python library, currently at version 0.13.0, enables other Python libraries and applications to dynamically load optimized compute kernels directly from the Hugging Face Kernel Hub. These kernels are designed to be portable, unique (multiple versions can coexist), and compatible across various Python and PyTorch configurations. The library supports a rapid release cycle, with frequent updates to add new features and improve performance.

Warnings

breaking Calling `get_kernel()` without specifying a `version` argument is deprecated in `kernels 0.12` and will become an error in `0.14` (except for local kernels).
Fix: Always pass the `version` argument to `get_kernel()` (e.g., `get_kernel('org/name', version=1)`). Consult kernel documentation for available versions.
deprecated Support for 'universal kernels' is being phased out in favor of 'noarch kernels'. Future versions will emit deprecation warnings, eventually leading to removal.
Fix: Kernel authors should migrate to building 'noarch kernels'. Users generally won't need to change their code, but should be aware of potential changes in kernel availability or performance if relying on older 'universal' builds.
gotcha Most optimized kernels from the Hugging Face Hub (especially `kernels-community`) are designed for GPU acceleration and require a compatible PyTorch installation (e.g., `torch>=2.5` with CUDA).
Fix: Ensure `torch` is installed with CUDA support (`pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu118` or similar for your CUDA version) and that you have a functional GPU environment. Attempting to use GPU-only kernels on a CPU-only setup will result in errors.

Install

pip install kernels Install with pip
pip install kernels torch>=2.5 Install with PyTorch (recommended for GPU kernels)

Imports

get_kernel
```
from kernels import get_kernel
```
This is the primary function for downloading and accessing kernels from the Hub.

Quickstart

This quickstart demonstrates how to fetch a pre-optimized activation kernel from the Hugging Face Hub and execute it using PyTorch. It highlights the importance of specifying the kernel version for stability and requires a CUDA-enabled environment for GPU execution.

import torch
from kernels import get_kernel
import os

# NOTE: For this example to run, you need PyTorch installed with CUDA support
# and a CUDA-enabled GPU. This example might fail on CPU-only setups.
# Ensure 'torch' is installed: pip install torch

# Download an optimized activation kernel from the Hugging Face Hub
# Specifying the version is important to avoid future breaking changes (see warnings)
try:
    activation = get_kernel("kernels-community/activation", version=1)
    print("Kernel downloaded successfully!")

    # Example usage: Generate a random tensor on a CUDA device
    if torch.cuda.is_available():
        x = torch.randn((10, 10), dtype=torch.float16, device="cuda")
        y = torch.empty_like(x)

        # Run the kernel (e.g., gelu_fast)
        activation.gelu_fast(y, x)
        print("Kernel executed successfully on CUDA!")
        # print(y)
    else:
        print("CUDA is not available. Skipping kernel execution on GPU.")

except Exception as e:
    print(f"An error occurred during quickstart: {e}")
    print("Please ensure 'torch' is installed with CUDA and you have a compatible GPU.")

view raw JSON →