Optuna Integration

4.8.0 · active · verified Sun Apr 12

Optuna Integration is a Python package that provides extended functionalities for Optuna, an automatic hyperparameter optimization software framework, in combination with various third-party machine learning libraries such as PyTorch, scikit-learn, and TensorFlow. It allows users to seamlessly integrate Optuna's powerful optimization capabilities and pruning mechanisms into their existing ML workflows. The library is currently at version 4.8.0, actively maintained, and follows Optuna's release cadence, typically with multiple releases per major Optuna version.

Warnings

breaking The integration modules have been migrated from the main `optuna` package to `optuna-integration`. Importing from `optuna.integration` is deprecated and will eventually be removed.
Fix: Update import statements from `from optuna.integration import ...` to `from optuna_integration import ...`.
breaking Optuna (and by extension, optuna-integration) dropped support for Python 3.8 starting with Optuna 4.0.0.
Fix: Ensure your Python environment is version 3.9 or higher.
deprecated The `verbosity` argument in `LightGBMTuner` has been removed. Use the `set_verbosity` method instead to control logging levels.
Fix: Replace `LightGBMTuner(..., verbosity=...)` with `tuner = LightGBMTuner(...)` followed by `tuner.set_verbosity(...)`.
gotcha When using `OptunaSearchCV` with `cv` (cross-validation), ensure the underlying estimator can handle the data splits without issues. Some estimators might require specific random states or data types.
Fix: Carefully review the documentation of the integrated estimator regarding its compatibility with scikit-learn's cross-validation utilities and potential data transformation needs.

Install

pip install optuna-integration Install core optuna-integration
pip install optuna-integration[sklearn] Install with scikit-learn dependencies
pip install optuna-integration[lightgbm] Install with LightGBM dependencies

Imports

OptunaSearchCV
```
from optuna_integration.sklearn import OptunaSearchCV
```
Integration modules have officially migrated from `optuna.integration` to `optuna_integration` for better separation and maintainability. While the old path might still work for backward compatibility, the new path is recommended.
LightGBMTuner
```
from optuna_integration.lightgbm import LightGBMTuner
```
As with `OptunaSearchCV`, LightGBM integration classes have moved to the `optuna_integration` package.

Quickstart

This quickstart demonstrates how to use `OptunaSearchCV` from `optuna_integration.sklearn` to perform hyperparameter optimization for a scikit-learn `SVC` estimator. It defines a search space for the 'C' and 'kernel' parameters and runs a specified number of trials. The example also implicitly shows how an objective function for Optuna works for direct `study.optimize` usage (though `OptunaSearchCV` abstracts this for scikit-learn models).

import optuna
from optuna_integration.sklearn import OptunaSearchCV
from sklearn.datasets import load_iris
from sklearn.svm import SVC
from sklearn.model_selection import train_test_split

def objective_svc(trial):
    svc_c = trial.suggest_float('svc_c', 1e-10, 1e10, log=True)
    svc_gamma = trial.suggest_float('svc_gamma', 1e-10, 1e10, log=True)
    classifier_obj = SVC(C=svc_c, gamma=svc_gamma)
    X, y = load_iris(return_X_y=True)
    X_train, X_test, y_train, y_test = train_test_split(X, y, random_state=0)
    classifier_obj.fit(X_train, y_train)
    return classifier_obj.score(X_test, y_test)

# Using OptunaSearchCV for a scikit-learn estimator
# This automatically creates an Optuna study and optimizes the hyperparameters
optuna_search = OptunaSearchCV(
    estimator=SVC(gamma='auto', random_state=0),
    param_distributions={
        'C': optuna.distributions.FloatDistribution(1e-10, 1e10, log=True),
        'kernel': ['linear', 'rbf']
    },
    n_trials=10,
    random_state=0,
    cv=3
)
X, y = load_iris(return_X_y=True)
optuna_search.fit(X, y)

print(f"Best parameters found by OptunaSearchCV: {optuna_search.best_params_}")
print(f"Best score found by OptunaSearchCV: {optuna_search.best_score_}")

view raw JSON →