{"id":23131,"library":"ms-swift","title":"ms-swift","description":"A scalable lightweight infrastructure for fine-tuning large language models (LLMs), vision-language models (VLMs), and embedding models. Supports LoRA, QLoRA, full fine-tuning, and reinforcement learning methods like DPO, GRPO, and PPO. Version 4.1.3 is the latest, with releases every few weeks.","status":"active","version":"4.1.3","language":"python","source_language":"en","source_url":"https://github.com/modelscope/ms-swift","tags":["fine-tuning","llm","lora","transformers","model-training","peft","deep-learning"],"install":[{"cmd":"pip install ms-swift","lang":"bash","label":"Install from PyPI"},{"cmd":"pip install 'ms-swift[llm]'","lang":"bash","label":"Install with LLM dependencies"},{"cmd":"pip install 'ms-swift[all]'","lang":"bash","label":"Install all extras"}],"dependencies":[{"reason":"Core dependency for model training and inference","package":"torch","optional":false},{"reason":"Hugging Face Transformers integration","package":"transformers","optional":false},{"reason":"Distributed training acceleration","package":"accelerate","optional":false},{"reason":"Dataset loading and processing","package":"datasets","optional":false},{"reason":"Parameter-efficient fine-tuning (LoRA, etc.)","package":"peft","optional":false},{"reason":"Experiment tracking (optional)","package":"wandb","optional":true}],"imports":[{"note":"Incorrect path before v4.0.0; SwiftTrainer moved to swift.trainers","wrong":"from swift import SwiftTrainer","symbol":"SwiffTrainer","correct":"from swift.trainers import SwiftTrainer"},{"note":"Dataset utilities moved to swift.utils in v4.0","wrong":"from swift.data import get_dataset","symbol":"DatasetName","correct":"from swift.utils import get_dataset"},{"note":"LoRAConfig is top-level since v4.0","wrong":"from swift.tuners import LoRAConfig","symbol":"LoRAConfig","correct":"from swift import LoRAConfig"}],"quickstart":{"code":"from swift import Swift, SwiftModel, LoRAConfig\nimport os\n\n# Load a pre-trained model and apply LoRA\nmodel = SwiftModel.from_pretrained(\"Qwen/Qwen2.5-1.5B\")\nlora_config = LoRAConfig(r=8, lora_alpha=32, target_modules=[\"q_proj\", \"v_proj\"])\nmodel = Swift.prepare_model(model, lora_config)\n\n# Quick training snippet (simplified)\nfrom swift.trainers import Seq2SeqTrainer\nfrom swift.utils import get_dataset\ntrain_dataset = get_dataset(\"json\", data_files=\"train.jsonl\")\ntrainer = Seq2SeqTrainer(model=model, train_dataset=train_dataset)\ntrainer.train()","lang":"python","description":"Basic LoRA fine-tuning with Qwen2.5. Demonstrates correct imports and usage."},"warnings":[{"fix":"Update imports: `from swift.trainers import SwiftTrainer`, `from swift.utils import get_dataset`. Use `from swift import LoRAConfig` instead of `from swift.tuners`.","message":"In v4.0.0, the package structure was refactored. SwiftTrainer moved from `swift` to `swift.trainers`, and many utility functions moved to `swift.utils`. Code written for v3.x will break on v4.x without import adjustments.","severity":"breaking","affected_versions":">=4.0.0"},{"fix":"If you rely on Megatron, use the new repo: https://github.com/modelscope/mcore-bridge","message":"The `Megatron` integration (Megatron-SWIFT) has been extracted to a separate repository `mcore-bridge` in v4.1.0. Future updates for Megatron training will be there.","severity":"deprecated","affected_versions":">=4.1.0"},{"fix":"Check model module names via `model.named_modules()` and set `target_modules` accordingly. For most models, `[\"q_proj\", \"v_proj\", \"k_proj\", \"o_proj\"]` works.","message":"When using `Swift.prepare_model`, the `target_modules` parameter in LoRAConfig must match actual module names in the model. Common mistake: using default target_modules that may not exist, leading to no LoRA applied silently.","severity":"gotcha","affected_versions":"all"},{"fix":"Use Python 3.10 or higher for full functionality.","message":"ms-swift requires Python >=3.8. However, some features (like GRPO) may need Python >=3.10 due to newer dependency versions. Using Python 3.8 may cause silent failures or missing optimizations.","severity":"gotcha","affected_versions":">=4.0"}],"env_vars":null,"last_verified":"2026-05-01T00:00:00.000Z","next_check":"2026-07-30T00:00:00.000Z","problems":[{"fix":"Use `from swift.trainers import SwiftTrainer` instead.","cause":"The import path changed in v4.0.0.","error":"ImportError: cannot import name 'SwiftTrainer' from 'swift'"},{"fix":"Install with `pip install ms-swift` and import as `import swift`.","cause":"The package `ms-swift` is installed but the module is `swift`. Some users try `import ms_swift` or `import msswift`.","error":"ModuleNotFoundError: No module named 'swift'"},{"fix":"Use `from swift import LoRAConfig` and pass `target_modules` as a list of strings.","cause":"Very old version (pre-1.0) used a different config structure. Or you are using `from swift.tuners import LoRAConfig` which might be deprecated.","error":"AttributeError: 'LoRAConfig' object has no attribute 'target_modules'"},{"fix":"Access the underlying model via `model.model` if needed, or use ms-swift's own training loop.","cause":"ms-swift wraps PEFT models in its own SwiftModel. Some utility functions expect the underlying model.","error":"RuntimeError: Expected a 'PeftModel' but got 'SwiftModel'"}],"ecosystem":"pypi","meta_description":null,"install_score":null,"install_tag":null,"quickstart_score":null,"quickstart_tag":null}