GPyTorch

Warnings

• breaking GPyTorch versions 1.14 and later require Python >= 3.10 and PyTorch >= 2.0. Attempting to install or run with older versions will lead to incompatibility issues.
  Fix: Ensure your Python environment is 3.10+ and PyTorch is 2.0+ before installing GPyTorch >= 1.14. You can check PyTorch compatibility at https://pytorch.org/get-started/locally/
• breaking A temporary breaking change was introduced in v1.14.1 related to the `LinearKernel`'s `ard_num_dims` property, which was quickly reverted in v1.14.2.
  Fix: If you are on v1.14.1, upgrade to v1.14.2 or a newer version to avoid this specific breaking change and benefit from the fix.
• deprecated The `gpytorch.random_variables` module and its classes (e.g., `GaussianRandomVariable`, `MultitaskGaussianRandomVariable`) were deprecated and replaced by `gpytorch.distributions`.
  Fix: Use classes from `gpytorch.distributions`, such as `gpytorch.distributions.MultivariateNormal` or `gpytorch.distributions.MultitaskMultivariateNormal`.
• gotcha The `jaxtyping` dependency was removed in v1.15.2. Users relying on `jaxtyping` for static type checking or runtime validation with GPyTorch might notice changes in type hint behavior or require updates to their type-checking configurations.
  Fix: Review your type-checking setup if you were explicitly using `jaxtyping` with GPyTorch. `jaxtyping` itself now supports PyTorch without a JAX dependency, so direct usage is still possible if desired.
• gotcha A potential bug with `gpytorch.settings.debug.on()` was fixed in v1.15.2, meaning its behavior might have been unreliable or incorrect in prior versions.
  Fix: Upgrade to v1.15.2 or later to ensure that `gpytorch.settings.debug.on()` functions as expected.

Install

• pip install gpytorch PyPI
• conda install gpytorch -c gpytorch Conda

Imports

• gpytorch

  import gpytorch

• ExactGP

  from gpytorch.models import ExactGP

• GaussianLikelihood

  from gpytorch.likelihoods import GaussianLikelihood

• ConstantMean

  from gpytorch.means import ConstantMean

• ScaleKernel

  from gpytorch.kernels import ScaleKernel

• RBFKernel

  from gpytorch.kernels import RBFKernel

• MultivariateNormal

  from gpytorch.distributions import MultivariateNormal

  gpytorch.random_variables was deprecated and replaced by gpytorch.distributions in early versions.

Quickstart

This quickstart demonstrates a simple exact Gaussian Process regression. It defines a GP model, a Gaussian likelihood, trains the model using the marginal log likelihood, and then makes predictions including confidence intervals.

import math
import torch
import gpytorch
from gpytorch.models import ExactGP
from gpytorch.likelihoods import GaussianLikelihood
from gpytorch.means import ConstantMean
from gpytorch.kernels import ScaleKernel, RBFKernel
from gpytorch.distributions import MultivariateNormal
from torch.optim import Adam

# 1. Set up training data
train_x = torch.linspace(0, 1, 100)
train_y = torch.sin(train_x * (2 * math.pi)) + torch.randn(train_x.size()) * math.sqrt(0.04)

# 2. Define the GP model
class ExactGPModel(ExactGP):
    def __init__(self, train_x, train_y, likelihood):
        super(ExactGPModel, self).__init__(train_x, train_y, likelihood)
        self.mean_module = ConstantMean()
        self.covar_module = ScaleKernel(RBFKernel())

    def forward(self, x):
        mean_x = self.mean_module(x)
        covar_x = self.covar_module(x)
        return MultivariateNormal(mean_x, covar_x)

# Initialize likelihood and model
likelihood = GaussianLikelihood()
model = ExactGPModel(train_x, train_y, likelihood)

# 3. Train the model
# Put model and likelihood in training mode
model.train()
likelihood.train()

# Use the Adam optimizer
optimizer = Adam(model.parameters(), lr=0.1)

# "Loss" for GPs - the marginal log likelihood
mll = gpytorch.mlls.ExactMarginalLogLikelihood(likelihood, model)

for i in range(50): # typically 50 training iterations
    optimizer.zero_grad()
    output = model(train_x)
    loss = -mll(output, train_y)
    loss.backward()
    optimizer.step()

# 4. Make predictions
model.eval()
likelihood.eval()

with torch.no_grad(), gpytorch.settings.fast_pred_var():
    test_x = torch.linspace(0, 1, 51)
    observed_pred = likelihood(model(test_x))
    mean = observed_pred.mean
    lower, upper = observed_pred.confidence_region()