Cache-DiT

Common errors

• Generated images have visible artifacts or lower quality.

  cause Overly aggressive caching settings (e.g., high `residual_diff_threshold`, too few `Fn_compute_blocks`, or insufficient inference steps) leading to accumulation of errors or poor feature reuse.
  fix Adjust `cache_config` parameters: Decrease `residual_diff_threshold` (e.g., to 0.12-0.15 from default 0.24), increase `Fn_compute_blocks` (e.g., to 8-12 from default 1), and ensure the `num_inference_steps` is adequate (e.g., >10 steps) to allow for proper warmup and cache effectiveness.

• `CACHE_DIT_BUILD_SVDQUANT=1` was set, but `nvcc` was not found. Activate the CUDA toolchain before building.

  cause The CUDA compiler (`nvcc`) is not found in the system's PATH or the active environment, which is required when attempting to build Cache-DiT with SVDQuant support from source.
  fix Ensure that your CUDA toolkit is correctly installed and its `bin` directory is in your system's PATH. If using `conda`, activate your CUDA-enabled environment (e.g., `conda activate cdit`) before running the pip install command.

• Performance Dashboard shows 0% cache hit?

  cause This typically means the model was not properly detected by the caching mechanism, inference steps were too short (e.g., <10 steps preventing proper warmup), or the caching mechanism was not successfully enabled.
  fix In ComfyUI, try manual preset selection for the model in the CacheDiT node. For Python, ensure `cache_dit.enable_cache()` was called successfully. Increase the number of inference steps in your sampler, as a 'warmup' phase (e.g., 3-4 steps) is usually required before caching becomes effective. Check the console logs for 'Lightweight cache enabled' message to confirm activation.

Warnings

• gotcha Aggressive caching settings can degrade output quality. Over-caching, particularly with high `residual_diff_threshold` or insufficient `Fn_compute_blocks`, can introduce artifacts or lower fidelity images.
  Fix: Monitor output quality and adjust `cache_config` parameters. Consider lowering `residual_diff_threshold` from the default (0.24) and increasing `Fn_compute_blocks` from the default (1) for better quality. Ensure sufficient inference steps are used.
• gotcha Using `cache-dit` with `torch.compile` and dynamic input shapes may trigger `recompile_limit` errors, leading to fallback to eager mode and negating performance benefits.
  Fix: Increase the `recompile_limit` for `torch._dynamo` if you encounter this issue. For example, `torch._dynamo.config.recompile_limit = 100`.
• gotcha FP8 quantization with tensor parallelism may encounter memory layout mismatch errors in certain layers. The `per_tensor_fallback` option addresses this.
  Fix: Set `per_tensor_fallback=True` (which is often the default) in `QuantizeConfig` when calling `enable_cache` to allow unsupported layers to fall back to FP8 per-tensor quantization, preventing errors.
• gotcha When integrating with ComfyUI, specific DiT models like LTX-2 or WAN2.2 14B MoE require dedicated optimizer nodes (e.g., `⚡ LTX2 Cache Optimizer`, `⚡ Wan Cache Optimizer`) instead of the general `⚡ CacheDiT Accelerator` node for optimal performance and quality.
  Fix: Refer to the `ComfyUI-CacheDiT` documentation and use the model-specific optimizer nodes for LTX-2 and WAN2.2 14B models.

Install

• pip install -U cache-dit Install stable release from PyPI

Imports

• cache_dit

  import cache_dit

• DBCacheConfig

  from cache_dit import DBCacheConfig, ParallelismConfig, QuantizeConfig

• ParallelismConfig

  from cache_dit import DBCacheConfig, ParallelismConfig, QuantizeConfig

• QuantizeConfig

  from cache_dit import DBCacheConfig, ParallelismConfig, QuantizeConfig

• enable_cache

  cache_dit.enable_cache(pipeline_instance)

Quickstart

To quickly get started, load a `DiffusionPipeline` model and apply `cache_dit.enable_cache()` to activate acceleration. You can customize caching, parallelism, and quantization through `DBCacheConfig`, `ParallelismConfig`, and `QuantizeConfig` respectively. The example demonstrates enabling hybrid acceleration with FP8 quantization and Ulysses parallelism.

import torch
from diffusers import DiffusionPipeline
import cache_dit
from cache_dit import DBCacheConfig, ParallelismConfig, QuantizeConfig

# Load a DiffusionPipeline model
pip = DiffusionPipeline.from_pretrained("runwayml/stable-diffusion-v1-5").to("cuda")

# Enable Cache-DiT acceleration with default cache, parallelism, and quantization
cache_dit.enable_cache(
    pip,
    cache_config=DBCacheConfig(), # Default cache settings
    parallelism_config=ParallelismConfig(ulysses_size=2), # Example: enable Ulysses parallelism
    quantize_config=QuantizeConfig(quant_type="float8_per_row") # Example: enable FP8 quantization
)

# Run inference as usual
prompt = "a photo of an astronaut riding a horse on mars"
image = pip(prompt).images[0]
image.save("astronaut_horse_mars.png")

# To disable cache_dit acceleration later:
# cache_dit.disable_cache(pip)