LightGBM

4.6.0 · active · verified Sun Apr 05

LightGBM (Light Gradient Boosting Machine) is an open-source, high-performance gradient boosting framework developed by Microsoft. It uses tree-based learning algorithms and is designed for efficiency, scalability, and high accuracy, particularly with large datasets. Key innovations like Gradient-based One-Side Sampling (GOSS) and Exclusive Feature Bundling (EFB) contribute to its faster training speeds and lower memory usage. The library is actively maintained, with frequent releases, and is currently at version 4.6.0.

Warnings

breaking LightGBM v4.x introduced significant breaking changes. Key updates include making `Booster` and `Dataset` `handle` attributes private, removal of a hard `scikit-learn` dependency (now optional), and switching to PEP 517/518 builds (removal of `setup.py`). Furthermore, `feature_name` and `categorical_feature` parameters should now be set on the `lgb.Dataset` object directly, not passed to `train()` or `cv()` functions. CUDA 10 support was dropped in favor of CUDA 12.
Fix: Review release notes for v4.0.0. Update code to use `lgb.Dataset.set_feature_name()` and `lgb.Dataset.set_categorical_feature()` or ensure features are correctly typed/named. Ensure `scikit-learn>=0.24.2` is installed for `LGBMClassifier/Regressor`.
gotcha LightGBM can handle categorical features natively, but they should be converted to integer types (e.g., 0, 1, 2...). Passing non-integer or excessively large integer values as categorical features can lead to warnings or unexpected behavior.
Fix: Ensure all categorical features are explicitly converted to `int` type (e.g., using `LabelEncoder` or pandas `astype('category').cat.codes`) before creating `lgb.Dataset` or fitting `LGBMClassifier/Regressor`. Values should ideally range from 0 to `num_categories - 1`.
gotcha LightGBM is prone to overfitting, especially on small datasets (<10,000 records) or with excessively deep trees.
Fix: Implement hyperparameter tuning for `num_leaves`, `max_depth`, `min_data_in_leaf`, and regularization parameters (`lambda_l1`, `lambda_l2`). Always use early stopping with a validation set during training. Consider `min_sum_hessian_in_leaf` and data/feature bagging (`bagging_fraction`, `feature_fraction`).
gotcha Using GPU acceleration requires specific setup beyond `pip install lightgbm`. While newer versions (v4.x) have improved CUDA support, you typically need OpenCL Runtime libraries. Some advanced GPU features or specific CUDA versions might require building from source.
Fix: For basic GPU usage, ensure OpenCL Runtime libraries are installed (often via GPU drivers). For specific CUDA versions or advanced GPU features, consult the official LightGBM installation guide for building from source or specialized wheels. Use `device='gpu'` in parameters.
gotcha On Linux, if LightGBM hangs when multithreading (OpenMP) and using forking (e.g., in multiprocessing scenarios), it's a known bug.
Fix: Set `nthreads=1` in your LightGBM parameters to disable LightGBM's internal multithreading when using forking mechanisms.

Install

pip install lightgbm Basic Installation
pip install 'lightgbm[pandas,scikit-learn,dask]' With Common Integrations
brew install libomp # For macOS users needing OpenMP macOS OpenMP Dependency

Imports

lightgbm
```
import lightgbm as lgb
```
LGBMClassifier
```
from lightgbm import LGBMClassifier
```
LGBMRegressor
```
from lightgbm import LGBMRegressor
```
Dataset
```
lgb_train = lgb.Dataset(X_train, y_train)
```
As of v4.0.0, 'feature_name' and 'categorical_feature' parameters should be set directly on the `Dataset` object or inferred, not passed to the constructor or `train`/`cv` functions.

Quickstart

This quickstart demonstrates how to train a binary classification model using LightGBM's scikit-learn compatible API (`LGBMClassifier`). It covers data preparation, model initialization, training with early stopping, prediction, and evaluation. For non-scikit-learn API, `lgb.Dataset` and `lgb.train` are used.

import numpy as np
import lightgbm as lgb
from sklearn.model_selection import train_test_split
from sklearn.metrics import accuracy_score

# Generate some dummy data
X = np.random.rand(1000, 10)
y = np.random.randint(0, 2, 1000)

# Split data into training and testing sets
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

# Initialize and train the LGBMClassifier
# Using scikit-learn API for convenience
model = lgb.LGBMClassifier(objective='binary', random_state=42)
model.fit(X_train, y_train,
          eval_set=[(X_test, y_test)],
          callbacks=[lgb.early_stopping(10)]) # Early stopping after 10 rounds without improvement

# Make predictions
y_pred = model.predict(X_test)

# Evaluate the model
accuracy = accuracy_score(y_test, y_pred)
print(f"Model Accuracy: {accuracy:.4f}")

view raw JSON →