SetFit

1.1.3 · active · verified Thu Apr 16

SetFit is a Python library for efficient few-shot learning using Sentence Transformers. It enables training accurate text classifiers with minimal labeled data by finetuning pre-trained Sentence Transformer models. The library is prompt-free, fast to train, and offers multilingual support. The current version is 1.1.3, and the project maintains an active release cadence with frequent patch updates addressing compatibility and minor fixes, alongside larger feature releases.

Common errors

```
AttributeError: 'SetFitModel' object has no attribute 'fit'
```
cause Attempting to use the `model.fit()` method, which was removed in SetFit v1.0.0.

fix The training API was refactored. Use `SetFitTrainer` with a `TrainingArguments` instance and call `trainer.train()` instead.
```
TypeError: SetFitTrainer.__init__() got an unexpected keyword argument 'num_epochs' (or 'batch_size', 'learning_rate', etc.)
```
cause Training hyperparameters (e.g., `num_epochs`, `batch_size`, `learning_rate`) were passed directly to `SetFitTrainer`'s constructor, but in v1.0.0 and later, they must be encapsulated within a `TrainingArguments` object.

fix Create an instance of `TrainingArguments` and pass these parameters to it, then provide the `TrainingArguments` instance to the `SetFitTrainer` via the `args` parameter.
```
RuntimeError: [enforce fail at inline_container.cc:471] PytorchStreamWriter failed writing file data/...
```
cause This error during `trainer.train()` often indicates a corrupted PyTorch cache or insufficient disk space.

fix Try clearing the PyTorch Sentence Transformers cache (`~/.cache/torch/sentence_transformers/`) and ensure your system has enough free disk space.
```
Model outputs the exact same label regardless of input or very low accuracy (e.g., 60-63%) on a classification task.
```
cause This can indicate overfitting to the majority class, severely imbalanced datasets, or 'collapsing' of the embedding model where it fails to distinguish between classes.

fix Inspect class distribution of your dataset and balance if necessary. Reduce `num_epochs` or `num_iterations`, adjust `learning_rate`, or use techniques like early stopping. Evaluate with metrics appropriate for imbalanced data (e.g., F1-score, precision, recall).

Warnings

breaking SetFit v1.0.0 introduced significant API changes, particularly for the `SetFitTrainer` and how training arguments are handled. The old `model.fit()` method was removed, and training parameters were moved into a `TrainingArguments` dataclass.
Fix: Migrate `model.fit()` calls to use `SetFitTrainer` and pass training hyperparameters via a `TrainingArguments` instance. Refer to the v1.0.0 migration guide for details.
deprecated Python 3.7 support was deprecated starting from SetFit v1.1.0.
Fix: Upgrade to Python 3.9 or newer. SetFit is tested on Python 3.9+.
gotcha Frequent compatibility issues can arise with newer versions of core dependencies like `datasets`, `transformers`, and `sentence-transformers`. Patch releases of SetFit (`v1.1.3`, `v1.1.2`) often address these specific issues.
Fix: Ensure you are using a compatible set of dependency versions. If encountering errors, try upgrading SetFit to the latest patch release or consult the official documentation/GitHub issues for known compatibility notes.
gotcha Reproducibility of trained models across different SetFit versions (e.g., v0.6.0 vs v1.0.3) can be challenging due to changes in random seed handling for the model's head initialization and different sentence pair sampling methods.
Fix: Ensure `transformers.trainer_utils.set_seed()` is called before `SetFitModel.from_pretrained()`. Be aware that minor differences in sampling logic between major versions might still lead to slightly different models.
gotcha The `report_to="none"` argument in `TrainingArguments` was sometimes ignored, leading to unexpected logging behavior if Weights & Biases or Tensorboard were installed.
Fix: This issue was fixed in v1.1.1. Upgrade to SetFit v1.1.1 or later to ensure `report_to="none"` is respected.

Install

pip install setfit Basic Installation
pip install torch --index-url https://download.pytorch.org/whl/cu118 && pip install setfit Installation with CUDA support (recommended for GPU)

Imports

SetFitModel
```
from setfit import SetFitModel
```
SetFitTrainer
```
from setfit import SetFitTrainer
```
TrainingArguments
```
from setfit import TrainingArguments
```

SupConLoss

wrong:

from setfit.modeling import SupConLoss

correct:

from setfit import SupConLoss

Path changed in v1.0.0

Quickstart

This quickstart demonstrates the typical workflow for training a SetFit model for text classification. It covers initializing a `SetFitModel` from the Hugging Face Hub, preparing a dataset (including simulating a few-shot scenario), configuring training parameters via `TrainingArguments`, creating and training a `SetFitTrainer`, evaluating the model, and making predictions. The example uses a small BGE model and the SST-2 dataset, sampling 8 examples per class for training.

from datasets import load_dataset
from setfit import SetFitModel, SetFitTrainer, TrainingArguments, sample_dataset
from sentence_transformers.losses import CosineSimilarityLoss

# 1. Initialize a SetFit model
model = SetFitModel.from_pretrained("BAAI/bge-small-en-v1.5")

# 2. Load and prepare a dataset (e.g., sst2 for sentiment classification)
dataset = load_dataset("SetFit/sst2")

# Simulate few-shot regime: 8 examples per class
train_dataset = sample_dataset(dataset["train"], label_column="label", num_samples=8)
eval_dataset = dataset["validation"]

# Optional: Map dataset columns if they are not 'text' and 'label'
column_mapping = {"sentence": "text", "label": "label"}

# 3. Define TrainingArguments
training_args = TrainingArguments(
    batch_size=16,
    num_iterations=20, # Number of text pairs to generate for contrastive learning
    num_epochs=1,      # Number of epochs to use for contrastive learning
    learning_rate=2e-5,
    seed=42,
    evaluation_strategy="epoch",
    save_strategy="epoch"
)

# 4. Create SetFitTrainer
trainer = SetFitTrainer(
    model=model,
    args=training_args,
    train_dataset=train_dataset,
    eval_dataset=eval_dataset,
    loss_class=CosineSimilarityLoss, # Loss function for contrastive learning
    metric="accuracy",
    column_mapping=column_mapping
)

# 5. Train the model
trainer.train()

# 6. Evaluate the model
metrics = trainer.evaluate()
print(f"Evaluation Metrics: {metrics}")

# 7. Make predictions
sentences = ["The movie was great!", "I didn't like the food."]
predictions = model.predict(sentences)
print(f"Predictions: {predictions}")

# 8. Push model to Hugging Face Hub (requires `huggingface_hub` login)
# trainer.push_to_hub("my-awesome-setfit-model")

view raw JSON →