CatBoost

Warnings

• breaking Support for Python 3.7 and Apache Spark 2.x was officially dropped in CatBoost version 1.2.8. Users relying on these older environments will need to upgrade their Python or Spark, or pin an older CatBoost version.
  Fix: Upgrade Python to 3.8+ or Spark to 3.x+. Alternatively, pin `catboost<1.2.8` in your project dependencies.
• gotcha For binary classification tasks, `model.predict_proba()` returns a 2D array of shape `(n_samples, 2)`, where columns represent probabilities for class 0 and class 1 respectively. Users commonly expect a 1D array of positive class probabilities.
  Fix: To get probabilities for the positive class (class 1), access the second column: `probabilities[:, 1]`.
• gotcha There was a period of flux regarding `numpy` 2.x compatibility. While `catboost>=1.2.8` now officially supports `numpy 2.x`, earlier versions (e.g., 1.2.6) explicitly prohibited it, which could cause dependency conflicts or runtime issues if `numpy` was upgraded prematurely.
  Fix: Ensure `catboost` is version `1.2.8` or newer for full `numpy 2.x` compatibility. If using an older `catboost` version, explicitly pin `numpy<2.0` in your environment.
• gotcha CatBoost's strength lies in its handling of categorical features, but they must be explicitly identified (e.g., using the `cat_features` parameter in `Pool` or `fit`). Failing to do so will cause them to be treated as numerical, potentially leading to suboptimal model performance or incorrect results.
  Fix: When creating a `Pool` object or calling `model.fit()`, pass a list of indices or names of your categorical columns to the `cat_features` argument.

Install

• pip install catboost CPU or GPU (if CUDA toolkit is installed)

Imports

• CatBoostClassifier

  from catboost import CatBoostClassifier

• CatBoostRegressor

  from catboost import CatBoostRegressor

• Pool

  from catboost import Pool

  The Pool object is crucial for efficient data handling, especially with categorical features or large datasets, ensuring optimal performance and correct feature interpretation.

Quickstart

This example demonstrates how to train a `CatBoostClassifier` on a synthetic dataset, correctly specifying categorical features using the `Pool` object. It then makes predictions and retrieves probabilities, highlighting CatBoost's typical workflow and efficient data handling.

from catboost import CatBoostClassifier, Pool
from sklearn.model_selection import train_test_split
from sklearn.datasets import make_classification

# Generate a synthetic dataset
X, y = make_classification(n_samples=1000, n_features=10, n_informative=5, n_redundant=0, random_state=42)
# Let's pretend the first two features are categorical (e.g., integer-encoded)
categorical_features_indices = [0, 1]

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

# Initialize CatBoostClassifier
model = CatBoostClassifier(
    iterations=100,
    learning_rate=0.1,
    depth=6,
    loss_function='Logloss',
    eval_metric='Accuracy',
    random_seed=42,
    verbose=False # Suppress training output for quickstart
)

# Create CatBoost Pool objects, explicitly specifying categorical features
train_pool = Pool(X_train, y_train, cat_features=categorical_features_indices)
test_pool = Pool(X_test, y_test, cat_features=categorical_features_indices)

# Train the model
model.fit(train_pool, eval_set=test_pool)

# Make predictions
predictions = model.predict(X_test)
probabilities = model.predict_proba(X_test)

print(f"Model accuracy on test set: {model.get_best_score()['validation']['Accuracy']:.4f}")
print(f"First 5 predictions: {predictions[:5].tolist()}")
print(f"First 5 probabilities (class 0, class 1): {probabilities[:5].tolist()}")