{"id":5152,"library":"clean-fid","title":"Clean-FID: Consistent FID Score Calculation","description":"Clean-FID is a Python library built on PyTorch for calculating the Fréchet Inception Distance (FID) and Kernel Inception Distance (KID) between image distributions. It addresses common inconsistencies in FID scores caused by differing or incorrect implementations of low-level image processing steps, such as resizing and quantization, in other libraries. The current version is 0.1.35, and it is actively maintained with periodic releases.","status":"active","version":"0.1.35","language":"en","source_language":"en","source_url":"https://github.com/GaParmar/clean-fid","tags":["FID","KID","Generative Models","PyTorch","Image Quality Metrics","Computer Vision"],"install":[{"cmd":"pip install clean-fid","lang":"bash","label":"Install stable version"}],"dependencies":[{"reason":"Core deep learning framework.","package":"torch","optional":false},{"reason":"Image datasets and models for PyTorch.","package":"torchvision","optional":false},{"reason":"Numerical computing.","package":"numpy","optional":false},{"reason":"Image processing.","package":"Pillow","optional":false},{"reason":"Scientific computing for statistical operations.","package":"scipy","optional":false},{"reason":"Progress bars.","package":"tqdm","optional":false},{"reason":"Fetching models and data.","package":"requests","optional":false},{"reason":"Used for efficient data storage/retrieval (e.g., cached features).","package":"lmdb","optional":true},{"reason":"Used for LPIPS metric, if enabled.","package":"lpips","optional":true},{"reason":"Build system, often used by PyTorch extensions.","package":"ninja","optional":true},{"reason":"For C extensions, if any.","package":"cython","optional":true},{"reason":"Multimedia processing, if needed for video datasets/inputs.","package":"ffmpeg","optional":true},{"reason":"Required for CLIP-FID calculations.","package":"clip @ git+https://github.com/openai/CLIP.git","optional":true},{"reason":"Only required for `mode=\"legacy_tensorflow\"` to reproduce TensorFlow-based legacy FID scores. May conflict with PyTorch CUDA installations.","package":"tensorflow-cpu","optional":true}],"imports":[{"note":"The primary interface for `clean-fid` is the `fid` module, which exposes functions like `compute_fid` and `compute_kid`.","symbol":"fid","correct":"from cleanfid import fid"}],"quickstart":{"code":"import os\nfrom cleanfid import fid\n\n# Create dummy image folders for demonstration\nif not os.path.exists('path_to_real_images'):\n    os.makedirs('path_to_real_images', exist_ok=True)\n    # In a real scenario, populate this folder with real images\n    # For this example, we'll just create a dummy file\n    with open('path_to_real_images/dummy_real.txt', 'w') as f: pass\n\nif not os.path.exists('path_to_generated_images'):\n    os.makedirs('path_to_generated_images', exist_ok=True)\n    # In a real scenario, populate this folder with generated images\n    # For this example, we'll just create a dummy file\n    with open('path_to_generated_images/dummy_gen.txt', 'w') as f: pass\n\n# Compute FID between two image folders\nfdir1 = \"path_to_real_images\"\nfdir2 = \"path_to_generated_images\"\n\n# Note: For actual FID computation, these folders need to contain actual images.\n# The dummy files above will cause an error when clean-fid tries to load images.\n# This is a placeholder for a runnable example structure.\n\n# Example of computing FID (will likely fail with dummy folders but shows API)\ntry:\n    score = fid.compute_fid(fdir1, fdir2)\n    print(f\"Computed FID: {score}\")\nexcept Exception as e:\n    print(f\"Error computing FID (expected with dummy data): {e}\")\n\n# Example with pre-computed dataset statistics (requires actual dataset name like 'FFHQ')\n# score_with_stats = fid.compute_fid(fdir2, dataset_name=\"FFHQ\", dataset_res=1024, dataset_split=\"trainval70k\")\n# print(f\"Computed FID with FFHQ stats: {score_with_stats}\")","lang":"python","description":"This quickstart demonstrates how to compute the FID score between two folders of images using `clean-fid`. Replace `path_to_real_images` and `path_to_generated_images` with actual directories containing your image files. The library also supports computing FID against pre-computed statistics for standard datasets or using a generative model directly."},"warnings":[{"fix":"Always use `clean-fid` for new evaluations or for comparing with 'clean' results. Be aware that scores from other libraries might not be directly comparable without using `clean-fid`'s legacy modes.","message":"FID scores from other libraries might be inconsistent. `clean-fid` aims to provide a 'clean' and reproducible FID by correctly implementing image resizing and quantization steps, which are often sources of discrepancy in other implementations (e.g., PyTorch-FID, TensorFlow-FID).","severity":"gotcha","affected_versions":"All versions"},{"fix":"For new, robust, and comparable evaluations, prefer the default `mode='clean'` (or omit the `mode` argument as 'clean' is default). Only use `legacy` modes when explicitly needing to match a specific prior work's (potentially flawed) FID calculation.","message":"Using `mode='legacy_pytorch'` or `mode='legacy_tensorflow'` will reproduce potentially inconsistent FID scores from older implementations. While useful for backward compatibility and comparison with existing literature, these modes do not reflect the 'clean' FID standard.","severity":"gotcha","affected_versions":"All versions"},{"fix":"Ensure consistent image compression (or lack thereof) across all image sets (real, generated, and any pre-computed statistics) for meaningful FID comparisons. When comparing with other works, verify their image compression practices.","message":"JPEG compression, even if perceptually subtle, can significantly impact FID scores. If your generated or real images are compressed (e.g., for storage efficiency), it can lead to different FID values compared to uncompressed images.","severity":"gotcha","affected_versions":"All versions"},{"fix":"If encountering installation issues with `clip`, try installing it manually before `clean-fid` or ensure your environment supports direct Git installs. Monitor the `openai/CLIP` repository for breaking changes if you rely on CLIP-FID functionality.","message":"The `clip` dependency for CLIP-FID is installed directly from GitHub (`clip @ git+https://github.com/openai/CLIP.git`). This direct Git installation can sometimes cause issues with dependency resolvers or break if the upstream repository changes in an incompatible way.","severity":"breaking","affected_versions":"All versions requiring CLIP-FID"}],"env_vars":null,"last_verified":"2026-04-13T00:00:00.000Z","next_check":"2026-07-12T00:00:00.000Z"}