{"id":2059,"library":"hf-transfer","title":"HF Transfer (hf-transfer)","description":"hf-transfer is a utility library designed to significantly speed up file transfers to and from the Hugging Face Hub. It works by optimizing the underlying transfer mechanisms used by the `huggingface_hub` library. The current version is 0.1.9, and it generally follows the release cadence of the broader Hugging Face ecosystem, with updates often tied to improvements in hub interaction.","status":"active","version":"0.1.9","language":"en","source_language":"en","source_url":"https://github.com/huggingface/hf_transfer","tags":["huggingface","transfer","acceleration","cli","hub","download","upload","performance"],"install":[{"cmd":"pip install hf-transfer","lang":"bash","label":"Install hf-transfer"}],"dependencies":[{"reason":"hf-transfer accelerates operations within huggingface_hub; it does not provide standalone transfer functionality.","package":"huggingface_hub","optional":false}],"imports":[{"note":"The primary interaction is by calling `hf_transfer.enable_transfer()` to patch `huggingface_hub`'s behavior.","wrong":"from hf_transfer import enable_transfer # This would work but generally not recommended for clarity when just enabling.","symbol":"enable_transfer","correct":"import hf_transfer\nhf_transfer.enable_transfer()"}],"quickstart":{"code":"import hf_transfer\nfrom huggingface_hub import snapshot_download\nimport os\n\n# Enable hf_transfer to accelerate subsequent Hugging Face Hub operations\nhf_transfer.enable_transfer()\n\n# Example: Download a small model from the Hugging Face Hub\n# This operation will now be accelerated by hf_transfer\nrepo_id = \"HuggingFaceH4/no_deid_test_data\"\nlocal_dir = \"./no_deid_test_data_hf_transfer\"\n\nprint(f\"Downloading {repo_id} to {local_dir} with hf_transfer acceleration...\")\n\ntry:\n    snapshot_download(repo_id=repo_id, local_dir=local_dir)\n    print(f\"Successfully downloaded {repo_id} to {local_dir}\")\nexcept Exception as e:\n    print(f\"An error occurred during download: {e}\")\nfinally:\n    # Clean up downloaded files for subsequent runs if desired\n    # import shutil\n    # if os.path.exists(local_dir):\n    #     shutil.rmtree(local_dir)\n    pass","lang":"python","description":"This quickstart demonstrates how to enable `hf_transfer` within your Python script and then use `huggingface_hub`'s `snapshot_download` function. `hf_transfer` will automatically accelerate the download operation once `hf_transfer.enable_transfer()` is called. Remember that `huggingface_hub` must also be installed for this to work."},"warnings":[{"fix":"Use `HF_TRANSFER_ENABLED=1` or `hf_transfer.enable_transfer()` in code. For older versions of `huggingface_hub` that might check the old variable, ensure compatibility or update `huggingface_hub`.","message":"The environment variable for enabling hf-transfer changed from `HF_HUB_ENABLE_HF_TRANSFER` to `HF_TRANSFER_ENABLED`.","severity":"breaking","affected_versions":"<=0.1.3 (old env var), >=0.1.4 (new env var)"},{"fix":"Call `hf_transfer.enable_transfer()` at the very beginning of your script, or set the `HF_TRANSFER_ENABLED=1` environment variable before running your Python script.","message":"hf-transfer must be enabled *before* any `huggingface_hub` operations are called for acceleration to take effect.","severity":"gotcha","affected_versions":"All versions"},{"fix":"Ensure `huggingface_hub` is installed (`pip install huggingface_hub`) alongside `hf-transfer`.","message":"hf-transfer does not provide standalone download/upload functionality; it acts as an accelerator for the `huggingface_hub` library. Therefore, `huggingface_hub` is a required dependency.","severity":"gotcha","affected_versions":"All versions"},{"fix":"Evaluate performance in your specific use case. If you experience issues, consider checking network configuration, firewall settings, and disk I/O performance.","message":"Performance benefits of hf-transfer are most noticeable for large files or many small files, and can be influenced by network conditions and local storage speed. It may not provide significant speedups for very small transfers.","severity":"gotcha","affected_versions":"All versions"}],"env_vars":null,"last_verified":"2026-04-09T00:00:00.000Z","next_check":"2026-07-08T00:00:00.000Z"}