Adapters

1.3.0 verified Fri May 01 auth: no python

A unified library for parameter-efficient and modular transfer learning, supporting adapter methods like LoRA, Bottleneck Adapters, AdapterPlus, DoRA, and ReFT. Built on Hugging Face Transformers. Current version: 1.3.0. Release cadence: irregular, roughly every 2-3 months.

pip install adapters

Common errors

error AttributeError: 'BertModel' object has no attribute 'add_adapter' ↓

cause Using a bare Hugging Face model instead of an AutoAdapterModel (or AdapterModel wrapper).

fix

Replace from transformers import BertModel with from adapters import AutoAdapterModel and use AutoAdapterModel.from_pretrained(...).

error ImportError: cannot import name 'AutoAdapterModel' from 'transformers' ↓

cause Trying to import AutoAdapterModel from transformers, but it is provided by the adapters package.

fix

Use from adapters import AutoAdapterModel.

error ValueError: Adapter 'my_adapter' not found. Make sure to add an adapter before training. ↓

cause Calling `train_adapter('my_adapter')` before adding the adapter with `add_adapter()`.

fix

Ensure model.add_adapter('my_adapter', config=...) is called before model.train_adapter('my_adapter').

Warnings

breaking Each version of adapters supports a specific range of transformers versions. Using an incompatible transformers version may cause import errors or silent misbehavior. Check the release notes for the supported transformers version. ↓

fix Pin transformers to the version listed in the adapters release (e.g., for adapters 1.3.0 use transformers >=4.51.0, <4.58.0).

gotcha When using QLoRA with bnb (bitsandbytes), the quantized model must be loaded with `load_in_4bit=True` and `torch_dtype=torch.bfloat16`. Omitting these may cause runtime errors or incorrect behavior. ↓

fix Use `model = AutoAdapterModel.from_pretrained(..., load_in_4bit=True, torch_dtype=torch.bfloat16)` and ensure bitsandbytes is installed.

gotcha The `adapter_to()` method is used to move adapter weights to a device and optionally convert dtype. Forgetting to call it after adding an adapter can leave weights on CPU, causing issues when training on GPU. ↓

fix After adding an adapter, call `model.adapter_to('cuda')` or `model.adapter_to('cuda', dtype=torch.float16)`.

Imports

AutoAdapterModel
```
from adapters import AutoAdapterModel
```
Standard import for loading any model with adapter support.
AdapterConfig
```
from adapters import AdapterConfig
```
Base class for configuration.
LoRAConfig
```
from adapters import LoRAConfig
```
LoRA adapter configuration.
setup_adapter_training
```
from adapters import setup_adapter_training
```
Utility to set up training with adapters.

Quickstart

Load a BERT model, add a LoRA adapter, and run a forward pass.

from transformers import AutoTokenizer
from adapters import AutoAdapterModel, LoRAConfig

tokenizer = AutoTokenizer.from_pretrained("bert-base-uncased")
model = AutoAdapterModel.from_pretrained("bert-base-uncased")

# Add a LoRA adapter with configuration
config = LoRAConfig(r=8, alpha=16)
model.add_adapter("my_lora_adapter", config=config)
model.train_adapter("my_lora_adapter")

# Example: encode a sentence
inputs = tokenizer("Hello, adapters!", return_tensors="pt")
outputs = model(**inputs)
print(outputs.last_hidden_state.shape)