Opacus

raw JSON →
1.5.4 verified Fri May 01 auth: no python

Opacus is a library for training PyTorch models with differential privacy. It supports per-sample gradient clipping, privacy accounting (including PRVAccountant), and fast gradient clipping. Current version is 1.5.4, compatible with Python >=3.7.5 and PyTorch 1.13+. It is actively maintained by Meta.

pip install opacus

Common errors

error AttributeError: 'PrivacyEngine' object has no attribute 'make_private' ↓
cause Using an older version of Opacus (pre-1.0) where the method was named differently.
fix
Upgrade to latest Opacus: pip install --upgrade opacus
error TypeError: cannot unpack non-iterable PrivacyEngine object ↓
cause Unpacking make_private() result into two variables but it returns three (model, optimizer, data_loader) since v1.5.
fix
Change to: model, optimizer, data_loader = privacy_engine.make_private(...)
error RuntimeError: DataLoader worker process exited before finishing ↓
cause Using the original data_loader instead of the one returned by make_private. Opacus replaces the sampler.
fix
Always use the data_loader returned by privacy_engine.make_private(). Never use the original DataLoader directly.
error ModuleNotFoundError: No module named 'opt_einsum' ↓
cause Opacus 1.5.0 and earlier required opt_einsum for linear layers; removed in 1.5.1+ but old installs may still need it.
fix
Upgrade to opacus>=1.5.1 or install opt_einsum: pip install opt_einsum

Warnings

gotcha DataLoader must be passed to make_private() and the returned loader used for training. Using the original loader breaks privacy. ↓
fix Always use the data_loader returned by privacy_engine.make_private() in training loops.
breaking In v1.5, the make_private() method signature changed to accept keyword arguments and returns three objects (model, optimizer, data_loader). Old code unpacking two items (e.g., model, optimizer = privacy_engine.make_private(...)) will break. ↓
fix Update to: model, optimizer, data_loader = privacy_engine.make_private(...). Read the v1.5 migration guide.
deprecated The old accountant interface via privacy_engine.accountant is deprecated. Use privacy_engine.get_epsilon(delta) directly. ↓
fix Replace privacy_engine.accountant.get_epsilon(delta) with privacy_engine.get_epsilon(delta).
gotcha Ghost clipping (for language models) requires a separate setup: use make_private_with_epsilon or specify ghost_clipping=True. Vanilla make_private does not apply ghost clipping. ↓
fix For LLM fine-tuning, use GhostClippingEngine or set ghost_clipping=True in make_private.

Imports

Quickstart

Minimal DP training loop with Opacus using PrivacyEngine.make_private.

import torch
from torch import nn
from torch.utils.data import DataLoader, TensorDataset
from opacus import PrivacyEngine

# Create a simple model
torch.manual_seed(0)
model = nn.Linear(10, 2)
data = torch.randn(64, 10)
labels = torch.randint(0, 2, (64,))
train_dataset = TensorDataset(data, labels)
train_loader = DataLoader(train_dataset, batch_size=32)

# Define optimizer and loss
optimizer = torch.optim.SGD(model.parameters(), lr=0.01)
criterion = nn.CrossEntropyLoss()

# Attach privacy engine
privacy_engine = PrivacyEngine()
model, optimizer, train_loader = privacy_engine.make_private(
    module=model,
    optimizer=optimizer,
    data_loader=train_loader,
    noise_multiplier=0.5,
    max_grad_norm=1.0,
)

# Training loop
for epoch in range(2):
    for x, y in train_loader:
        outputs = model(x)
        loss = criterion(outputs, y)
        loss.backward()
        optimizer.step()
        optimizer.zero_grad()

# Get privacy spent
epsilon = privacy_engine.get_epsilon(delta=1e-5)
print(f"Privacy spent: epsilon = {epsilon:.2f}")
last verified Fri May 01
next check Thu Jul 30