scikit-optimize (skopt)

0.10.2 · active · verified Sun Apr 12

Scikit-Optimize, often referred to as skopt, is a simple and efficient Python library for sequential model-based optimization. It's designed to minimize expensive and noisy black-box functions, building on top of NumPy, SciPy, and Scikit-Learn. Version 0.10.2 is the current release. The library is under active development, with releases occurring periodically, making it a robust tool for tasks like hyperparameter tuning in machine learning.

Warnings

gotcha Despite being actively developed, `scikit-optimize` has previously been described as 'experimental and under heavy development'. This can imply that API stability, especially between minor versions, might not be as rigid as more mature libraries, potentially leading to breaking changes.
Fix: Always pin exact versions (`scikit-optimize==X.Y.Z`) in production environments and review changelogs carefully when upgrading, particularly for minor version bumps.
gotcha Reproducibility of optimization runs depends on setting the `random_state` parameter consistently across all components that use randomness (e.g., `gp_minimize`, `Optimizer`, base estimators in `BayesSearchCV`). Failing to do so can lead to different results across runs.
Fix: Pass an integer to the `random_state` parameter in all relevant functions and classes (e.g., `gp_minimize(..., random_state=42)`).
gotcha The library offers two main interfaces: direct minimization functions (e.g., `gp_minimize`) for complete optimization loops, and the `Optimizer` class for an 'ask-and-tell' interface, providing more fine-grained control over the optimization process. Confusing these or misapplying the `ask-and-tell` pattern can lead to incorrect or inefficient optimization loops.
Fix: For simple, self-contained optimization, use `gp_minimize` or similar functions. For custom loops, parallel evaluations, or dynamic stopping conditions, use the `Optimizer` class with its `ask()` and `tell()` methods, understanding how to manage the state.
gotcha As `scikit-optimize` is built on top of NumPy, SciPy, and Scikit-learn, version incompatibilities with these underlying libraries can occur. Outdated or incompatible versions of these dependencies might cause installation issues, runtime errors, or unexpected behavior.
Fix: Ensure that your environment uses compatible versions of `numpy`, `scipy`, and `scikit-learn`. Refer to the `scikit-optimize` documentation or `setup.py` for recommended dependency versions. Using a fresh virtual environment or `conda-forge` for installation often helps manage these dependencies.

Install

pip install scikit-optimize Basic Installation
pip install scikit-optimize[plots] Installation with Plotting Dependencies
conda install conda-forge::scikit-optimize Conda Installation

Imports

gp_minimize
```
from skopt import gp_minimize
```
forest_minimize
```
from skopt import forest_minimize
```
Optimizer
```
from skopt import Optimizer
```
While 'skopt.optimizer' exists, the top-level import 'from skopt import Optimizer' is the commonly documented and simpler approach for direct use.
BayesSearchCV
```
from skopt import BayesSearchCV
```
BayesSearchCV is exposed directly at the top level of the skopt package for convenience, despite residing within a submodule.

Quickstart

This quickstart demonstrates how to use `gp_minimize` to find the minimum of a noisy black-box function within a defined search space. It sets up a simple 1D objective function and then applies Gaussian Process-based Bayesian optimization. The `random_state` ensures reproducibility.

import numpy as np
from skopt import gp_minimize

def f(x):
    # An example objective function to minimize
    # In a real scenario, this could be a machine learning model training and evaluation
    return (np.sin(5 * x[0]) * (1 - np.tanh(x[0] ** 2)) * 
            np.random.randn() * 0.1 + (x[0] - 0.5)**2)

# Define the search space: a single dimension from -2.0 to 2.0
space = [(-2.0, 2.0)]

# Perform Bayesian optimization using Gaussian Processes
# n_calls: total number of objective evaluations
# n_random_starts: number of random points to sample before fitting the surrogate model
# random_state: for reproducibility
res = gp_minimize(f, space, n_calls=20, n_random_starts=5, random_state=123)

print(f"Optimal value found: x*={res.x[0]:.4f}, f(x*)={res.fun:.4f}")

view raw JSON →