BoTorch: Bayesian Optimization in PyTorch

0.17.2 · active · verified Mon Apr 13

BoTorch (pronounced "bow-torch") is a library for Bayesian Optimization research built on top of PyTorch, leveraging auto-differentiation, GPU support, and a dynamic computation graph. It provides a modular and extensible interface for composing Bayesian Optimization primitives like models, acquisition functions, and optimizers. Currently at version 0.17.2, it is under active development with frequent maintenance and feature releases.

Warnings

breaking BoTorch v0.17.0+ requires Python >=3.11 and PyTorch >=2.2. Ensure your environment meets these minimum versions before upgrading.
Fix: Upgrade Python to 3.11+ and PyTorch to 2.2+. Check the BoTorch `CHANGELOG.md` or official documentation for precise version requirements with each new minor release.
breaking BoTorch v0.17.1+ requires GPyTorch >=1.15.2 and `linear_operator >=0.6.1`. Older versions of these dependencies will cause compatibility issues.
Fix: Update GPyTorch to >=1.15.2 and `linear_operator` to >=0.6.1.
deprecated The `qExpectedImprovement` acquisition function has known numerical issues and is strongly recommended to be replaced by `qLogExpectedImprovement` for improved stability and performance.
Fix: Replace `qExpectedImprovement` with `qLogExpectedImprovement` in your code. They share the same API.
breaking Several APIs were removed in BoTorch v0.17. These include `get_fitted_map_saas_ensemble`, `qMultiObjectiveMaxValueEntropy`, `FullyBayesianPosterior`, the `task_feature` parameter from `SingleTaskGP.construct_inputs`, and the `fixed_features` argument from `optimize_acqf_homotopy`.
Fix: Consult the `CHANGELOG.md` for specific replacements or alternative approaches. Plan for refactoring code that uses these removed components.
gotcha BoTorch is a low-level API for Bayesian Optimization research. For general-purpose Bayesian Optimization and experiment management, users not actively doing research on BO are recommended to use Ax, which provides a user-friendly interface on top of BoTorch.
Fix: Consider starting with Ax (Meta's Adaptive Experimentation platform) if you need a simpler, more managed BO workflow. If custom models or acquisition functions are needed, these can be plugged into Ax.
breaking The default hyperparameter priors for most models were updated in v0.12.0 to use dimension-scaled log-normal priors. This significantly improves robustness to dimensionality but might alter results for models fitted with older versions.
Fix: Review existing models for potential changes in optimization behavior. If consistent results with older versions are critical, carefully check model initialization or pin to an earlier BoTorch version.

Install

pip install botorch Install latest stable release

Imports

SingleTaskGP
```
from botorch.models import SingleTaskGP
```
LogExpectedImprovement
```
from botorch.acquisition import LogExpectedImprovement
```
Use LogExpectedImprovement (qLogEI) for numerical stability and improved optimization performance, as qEI has known numerical issues.

fit_gpytorch_mll

from botorch.fit import fit_gpytorch_mll

ExactMarginalLogLikelihood

from gpytorch.mlls import ExactMarginalLogLikelihood

optimize_acqf
```
from botorch.optim import optimize_acqf
```

Quickstart

This quickstart demonstrates a basic Bayesian Optimization loop with BoTorch: initializing training data, fitting a Gaussian Process model, constructing an acquisition function (LogExpectedImprovement for numerical stability), and optimizing it to propose the next best candidate.

import torch
from botorch.models import SingleTaskGP
from botorch.acquisition import LogExpectedImprovement
from botorch.fit import fit_gpytorch_mll
from gpytorch.mlls import ExactMarginalLogLikelihood
from botorch.optim import optimize_acqf
from botorch.models.transforms import Normalize, Standardize

# 1. Define objective function (e.g., a simple 2D function)
def objective_function(x):
    return 1 - (x - 0.5).norm(dim=-1, keepdim=True)

# 2. Generate initial training data
train_X = torch.rand(10, 2, dtype=torch.double) * 2
train_Y = objective_function(train_X)
train_Y += 0.1 * torch.randn_like(train_Y) # Add some noise

# 3. Fit a Gaussian Process model
gp = SingleTaskGP(
    train_X=train_X,
    train_Y=train_Y,
    input_transform=Normalize(d=2),
    outcome_transform=Standardize(m=1),
)
mll = ExactMarginalLogLikelihood(gp.likelihood, gp)
fit_gpytorch_mll(mll)

# 4. Construct an acquisition function
# Use LogExpectedImprovement for better numerical stability
log_ei = LogExpectedImprovement(model=gp, best_f=train_Y.max())

# 5. Optimize the acquisition function to get the next candidate
bounds = torch.stack([torch.zeros(2), torch.ones(2)]).to(torch.double)
candidate, acq_value = optimize_acqf(
    acq_function=log_ei,
    bounds=bounds,
    q=1,
    num_restarts=5,
    raw_samples=20,
)

print(f"Next candidate: {candidate}")
print(f"Acquisition function value at candidate: {acq_value}")

# (Optional) Evaluate the new candidate and update the model in a loop
# new_X = candidate
# new_Y = objective_function(new_X)
# train_X = torch.cat([train_X, new_X])
# train_Y = torch.cat([train_Y, new_Y])
# ... refit model ...

view raw JSON →