{"id":6919,"library":"torchtune","title":"PyTorch Native Library for LLM Fine-tuning","description":"torchtune is a PyTorch-native library designed for authoring, fine-tuning, and experimenting with Large Language Models (LLMs). It provides hackable training recipes for techniques like SFT, LoRA, QLoRA, FSDP, DPO, PPO, and QAT, supporting popular architectures such as Llama, Gemma, Mistral, Phi, and Qwen. While it offers a componentized design, memory efficiency, and strong integrations, active feature development for torchtune officially ceased in July 2025. The library will receive critical bug fixes and security patches through 2025, but no new features will be added, as the PyTorch team is developing a new product.","status":"maintenance","version":"0.6.1","language":"en","source_language":"en","source_url":"https://github.com/pytorch/torchtune","tags":["LLM","fine-tuning","PyTorch","AI","machine-learning","deep-learning","transformers","LoRA","QLoRA","FSDP","DPO"],"install":[{"cmd":"pip install torch torchvision torchao","lang":"bash","label":"PyTorch prerequisites"},{"cmd":"pip install torchtune","lang":"bash","label":"Install torchtune"}],"dependencies":[{"reason":"Core deep learning framework.","package":"torch","optional":false},{"reason":"Required for multimodal LLMs.","package":"torchvision","optional":true},{"reason":"For quantization APIs and techniques.","package":"torchao","optional":true},{"reason":"For low-memory optimizers, particularly with QLoRA/full finetuning on a single device. May not be available for non-CUDA platforms.","package":"bitsandbytes","optional":true},{"reason":"For accessing model weights from Hugging Face Hub.","package":"huggingface_hub","optional":true},{"reason":"For accessing training and evaluation datasets from Hugging Face Datasets.","package":"datasets","optional":true},{"reason":"For logging metrics and checkpoints to Weights & Biases.","package":"wandb","optional":true},{"reason":"For logging metrics and checkpoints to Comet ML.","package":"comet_ml","optional":true}],"imports":[{"symbol":"lora_llama2_7b","correct":"from torchtune.models.llama2 import lora_llama2_7b"},{"symbol":"Message","correct":"from torchtune.data import Message"},{"symbol":"InputOutputToMessages","correct":"from torchtune.data import InputOutputToMessages"},{"note":"Checkpointer classes moved under `torchtune.training` in recent versions.","wrong":"from torchtune.checkpointers import FullModelHFCheckpointer","symbol":"FullModelHFCheckpointer","correct":"from torchtune.training import FullModelHFCheckpointer"}],"quickstart":{"code":"# Ensure you have a Hugging Face token (HF_TOKEN) set as an environment variable or pass it directly.\n# export HF_TOKEN=\"hf_YOUR_TOKEN\"\nimport os\n\n# 1. Download Llama2 7B model weights and tokenizer\n# You need access to Meta Llama models on Hugging Face.\n# This command will download files to /tmp/Llama-2-7b-hf\nprint(\"Downloading model...\")\nos.system(\n    f\"tune download meta-llama/Llama-2-7b-hf --output-dir /tmp/Llama-2-7b-hf \"\n    f\"--ignore-patterns \\\"*.bin\\\" --hf-token {os.environ.get('HF_TOKEN', 'YOUR_HF_TOKEN_HERE')}\"\n)\n\n# 2. Copy a LoRA fine-tuning config for single device\nprint(\"Copying config...\")\nos.system(\"tune cp llama2/7B_lora_single_device ./my_llama2_lora_config.yaml\")\n\n# (Optional) Modify my_llama2_lora_config.yaml if needed, e.g., adjust batch_size,\n# dataset path, or output_dir. Ensure tokenizer path points to the downloaded model.\n# Example: Update output_dir to a persistent location and ensure tokenizer path is correct.\n\n# 3. Run the LoRA fine-tuning recipe\nprint(\"Running fine-tuning...\")\nos.system(\n    \"tune run lora_finetune_single_device --config ./my_llama2_lora_config.yaml \"\n    \"checkpointer.checkpoint_dir=/tmp/Llama-2-7b-hf \"\n    \"tokenizer.path=/tmp/Llama-2-7b-hf/tokenizer.model \"\n    \"output_dir=/tmp/torchtune_output\"\n)\n\nprint(\"Fine-tuning command executed. Check /tmp/torchtune_output for results.\")","lang":"python","description":"This quickstart demonstrates how to use the `tune` CLI to download a Llama2 7B model from Hugging Face Hub (requiring an `HF_TOKEN`), copy a built-in LoRA single-device fine-tuning configuration, and then execute the fine-tuning process. This is the primary way to interact with torchtune for running pre-defined recipes. Ensure PyTorch and torchtune are installed and replace 'YOUR_HF_TOKEN_HERE' or set the HF_TOKEN environment variable."},"warnings":[{"fix":"Plan for migration to a future PyTorch offering or ensure current needs are met by the existing feature set and bug fixes. Regularly check PyTorch's official announcements.","message":"Active feature development for torchtune officially ceased in July 2025. The library will receive critical bug fixes and security patches during 2025, but no new features will be added. Users are advised to be aware of this and monitor announcements from the PyTorch team regarding a successor product.","severity":"breaking","affected_versions":">=0.6.1"},{"fix":"Reduce batch size, enable activation checkpointing, use LoRA/QLoRA for parameter-efficient fine-tuning, or use `bf16` precision. Refer to torchtune's memory optimization documentation for specific guidance.","message":"Memory issues, particularly 'CUDA out of memory', are common when fine-tuning large LLMs. torchtune offers various memory optimizations, but careful configuration is required.","severity":"gotcha","affected_versions":"All versions"},{"fix":"Set the `HF_TOKEN` environment variable or pass it directly via `--hf-token` to the `tune download` command. Obtain a token from your Hugging Face settings page.","message":"Downloading gated models from Hugging Face Hub (e.g., Llama models) requires a valid Hugging Face authentication token (`HF_TOKEN`). Without it, download commands will fail.","severity":"gotcha","affected_versions":"All versions"},{"fix":"Refer to official examples and GitHub issues for common patterns. Expect to consult the source code or community forums for advanced customization of configs and recipes. The team has acknowledged and is working on improving documentation.","message":"The documentation around custom datasets and generation, especially with YAML configurations, has been a source of frustration for users due to a lack of clarity, requiring trial and error or reading the library's source code.","severity":"gotcha","affected_versions":"<=0.6.1"}],"env_vars":null,"last_verified":"2026-04-15T00:00:00.000Z","next_check":"2026-07-14T00:00:00.000Z","problems":[]}