Skrub

0.8.0 · active · verified Thu Apr 16

Skrub is a Python library for machine learning with dataframes, offering robust tools for cleaning, preprocessing, and encoding tabular data, particularly for heterogeneous or messy datasets. It provides scikit-learn compatible transformers and a powerful DataOps API for complex data pipelines. The current version is 0.8.0, with regular minor and patch releases.

Common errors

```
ImportError: cannot import name 'ApplyToCols' from 'skrub'
```
cause The `ApplyToCols` transformer class was removed in skrub version 0.8.0.

fix Update your code to remove imports of `ApplyToCols` and refactor the transformation logic using other `skrub` transformers or direct pandas operations, as the old class no longer exists.
```
DeprecationWarning: Ken embeddings are deprecated and will be removed in a future version.
```
cause Your code is using `skrub.KenEmbeddings`, which has been deprecated since skrub 0.6.2.

fix Replace `KenEmbeddings` with alternative encoders like `GapEncoder` or `TableVectorizer`. For example, `GapEncoder` often provides similar or better performance.
```
Your Python version is 3.9.x, but skrub >=0.7.0 requires Python >= 3.10. Please upgrade your Python version.
```
cause Skrub versions 0.7.0 and later enforce a minimum Python version of 3.10.

fix Upgrade your Python environment to version 3.10 or newer. Check `python --version` and install a newer version if needed.
```
AttributeError: 'DropCols' object has no attribute 'columns_'
```
cause Attribute names for `DropCols` and `SelectCols` instances were renamed in 0.7.1 for consistency (e.g., `columns_` might have been renamed to `cols_to_drop_` or `cols_to_select_`).

fix Review the skrub 0.7.1 release notes or current documentation for `DropCols` and `SelectCols` to identify the correct attribute names. For example, `drop_cols_` and `select_cols_` are common patterns.

Warnings

breaking The `ApplyToCols` and `ApplyToFrame` transformers have been removed in version 0.8.0. Their functionality is now intended to be covered by other methods or a simplified `ApplyToCols` (if a new one was introduced under a different path).
Fix: Remove imports and usages of `ApplyToCols` and `ApplyToFrame`. Consult the skrub 0.8.0 documentation for alternative strategies for column-wise transformations.
breaking Minimum Python version increased to 3.10 in skrub 0.7.0. Installing or running on older Python environments will fail.
Fix: Ensure your Python environment is 3.10 or newer. Upgrade Python using your preferred package manager (e.g., `conda install python=3.10` or `pyenv install 3.10.12`).
breaking Minimum versions for key dependencies `scikit-learn` (>=1.4.2), `requests` (>=2.27.1) were increased in 0.7.0. Additionally, the minimum `polars` version (optional dependency) increased to >=1.5 in 0.8.0.
Fix: Upgrade your `scikit-learn`, `requests`, and `polars` (if used) installations to meet the new minimum requirements: `pip install -U scikit-learn requests 'polars>=1.5'`.
deprecated Ken embeddings (`skrub.KenEmbeddings`) were deprecated in skrub 0.6.2 and will be removed in a future version. Usage will emit a `DeprecationWarning`.
Fix: Migrate away from `KenEmbeddings` to other available encoders provided by skrub, such as `GapEncoder` or `TableVectorizer`, which offer similar or enhanced functionality.
gotcha The `compute_ngram_distance` utility function was made private (`_compute_ngram_distance`) in 0.7.2 to reduce API clutter and indicate it's not part of the public API.
Fix: If you were directly using `compute_ngram_distance`, switch to its private counterpart `_compute_ngram_distance`. Be aware that private functions may change without notice.

Install

pip install skrub Install base library
pip install 'skrub[polars,all]' Install with optional dependencies for Polars and all features

Imports

GapEncoder
```
from skrub import GapEncoder
```
TableVectorizer
```
from skrub import TableVectorizer
```
StringEncoder
```
from skrub import StringEncoder
```
DataOps
wrong:
```
from skrub.dataops import DataOps
```
correct:
```
from skrub import DataOps
```
DataOps was promoted to top-level import in 0.6.0. The old path still works but top-level is preferred.
MinHashEncoder
```
from skrub import MinHashEncoder
```

Quickstart

This quickstart demonstrates how to use `skrub.TableVectorizer` within a scikit-learn pipeline to automatically preprocess a DataFrame containing mixed data types (here, messy categorical text) and then train a logistic regression model. `TableVectorizer` intelligently applies appropriate encoders to different column types.

import pandas as pd
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import train_test_split
from sklearn.pipeline import Pipeline
from skrub import TableVectorizer

# Sample DataFrame with messy categorical data
df = pd.DataFrame({
    'city': ['Paris', 'london', 'New-York', 'paris', 'tokyo', 'new york'],
    'country': ['France', 'United Kingdom', 'USA', 'France', 'Japan', 'United States'],
    'price': [100, 150, 200, 110, 180, 210]
})

X = df[['city', 'country']]
y = (df['price'] > 150).astype(int) # Binary target

X_train, X_test, y_train, y_test = train_test_split(X, y, random_state=42)

# Create a scikit-learn pipeline with TableVectorizer
pipeline = Pipeline([
    ('table_vectorizer', TableVectorizer(low_memory=True)), # Automatically handles different column types
    ('classifier', LogisticRegression(random_state=42))
])

# Fit and evaluate the pipeline
pipeline.fit(X_train, y_train)
score = pipeline.score(X_test, y_test)
print(f"Pipeline score: {score:.2f}")

view raw JSON →