XGBoost

Warnings

• breaking In XGBoost 2.0, several parameters related to GPU usage (`gpu_id`, `gpu_hist`, `gpu_predictor`, `cpu_predictor`, `gpu_coord_descent`) were replaced by a single `device` parameter. The `hist` tree method also became the default. The `predictor` parameter was removed.
  Fix: Migrate deprecated GPU-related parameters to the new `device` parameter (e.g., `device='cuda'` for GPU). Review and update code that explicitly set `tree_method` or relied on the old `predictor` parameter.
• breaking XGBoost 2.0 introduced breaking changes in prediction functions, renaming all data parameters to `X` for better scikit-learn estimator interface compliance. It also dropped the generation of pseudo-feature names for `np.ndarray` inputs to `DMatrix` and changed the default evaluation metric for `binary:logitraw` to `logloss`.
  Fix: Update calls to prediction functions to use `X` for data arguments. If your code relied on auto-generated feature names from NumPy arrays, you might need to provide them explicitly. Verify model evaluation logic, especially for binary classification with raw logistic output.
• breaking XGBoost 3.0 removed the deprecated `DeviceQuantileDMatrix`, dropped support for saving models in certain deprecated formats (though loading old models is still supported), and removed support for legacy (blocking) CUDA streams. It also now requires CUDA 12.0 or later.
  Fix: Remove usages of `DeviceQuantileDMatrix`. If you have code saving models in old formats, consider updating to current supported formats. Ensure your CUDA environment meets the minimum requirement of 12.0+ if using GPU acceleration.
• deprecated The `use_label_encoder` parameter in `XGBClassifier` and `XGBRegressor` was deprecated and is now effectively ignored (or will raise warnings/errors in older versions). Modern XGBoost handles label encoding internally.
  Fix: Remove `use_label_encoder=True/False` from `XGBClassifier` or `XGBRegressor` initializations. If you encounter warnings in older code, setting `use_label_encoder=False` was a common workaround.
• gotcha Blindly using default hyperparameters without tuning for your specific dataset is a common mistake, leading to sub-optimal model performance or overfitting/underfitting. XGBoost offers a wide range of parameters that need careful tuning.
  Fix: Always perform hyperparameter tuning using techniques like Grid Search, Random Search, or more advanced optimization frameworks (e.g., Optuna) tailored to your dataset's characteristics. Focus on `learning_rate`, `max_depth`, `min_child_weight`, `subsample`, `colsample_bytree`, and `gamma`.
• gotcha Ignoring class imbalance in your dataset can lead to models that perform poorly on the minority class, which is often the class of most interest. XGBoost provides mechanisms to address this.
  Fix: For binary classification, use the `scale_pos_weight` parameter. For multi-class problems, consider `class_weight`. Supplement these with sampling techniques like SMOTE if necessary.

Install

• pip install xgboost Install stable version

Imports

• xgboost

  import xgboost as xgb

• XGBClassifier

  from xgboost import XGBClassifier

• XGBRegressor

  from xgboost import XGBRegressor

• DMatrix

  import xgboost as xgb
  dtrain = xgb.DMatrix(data, label=label)

  While DMatrix can be imported directly, the conventional and often safer practice (especially with older versions or in complex setups) is to access it via the 'xgb' alias, i.e., xgb.DMatrix. Direct import can sometimes lead to issues in specific environments or when mixing with other XGBoost interfaces.

Quickstart

This quickstart demonstrates how to use XGBoost for a multi-class classification task. It shows both the scikit-learn compatible API (XGBClassifier) for ease of integration and the native XGBoost API (xgb.DMatrix, xgb.train) for more fine-grained control. The Iris dataset is used for simplicity, showcasing data loading, splitting, model training, prediction, and evaluation.

import xgboost as xgb
from sklearn.model_selection import train_test_split
from sklearn.datasets import load_iris
from sklearn.metrics import accuracy_score

# 1. Load data
iris = load_iris()
X, y = iris.data, iris.target

# 2. Split data into training and testing sets
X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.2, random_state=42
)

# 3. Initialize and train an XGBoost Classifier
model = xgb.XGBClassifier(objective='multi:softprob',  # Multi-class classification with probability output
                          num_class=len(iris.target_names),
                          eval_metric='mlogloss',
                          use_label_encoder=False # Suppress deprecation warning in older versions
                         )
model.fit(X_train, y_train)

# 4. Make predictions
y_pred = model.predict(X_test)

# 5. Evaluate the model
accuracy = accuracy_score(y_test, y_pred)
print(f"Accuracy: {accuracy:.2f}")

# Native API example (alternative to Scikit-learn API)
dtrain = xgb.DMatrix(X_train, label=y_train)
dtest = xgb.DMatrix(X_test, label=y_test)

params = {
    'objective': 'multi:softprob',
    'num_class': len(iris.target_names),
    'eval_metric': 'mlogloss'
}

num_round = 100
bst = xgb.train(params, dtrain, num_round, evals=[(dtest, 'test')])
preds_native = bst.predict(dtest)
# For multi:softprob, preds_native are probabilities, need argmax for class labels
preds_class_native = [p.argmax() for p in preds_native]
accuracy_native = accuracy_score(y_test, preds_class_native)
print(f"Native API Accuracy: {accuracy_native:.2f}")