TabPFN: Foundation model for tabular data

7.1.1 · active · verified Thu Apr 16

TabPFN is a transformer-based foundation model for tabular data that leverages prior-data based learning to achieve strong performance on small-to-medium sized datasets without requiring task-specific training. Currently at version 7.1.1, it is actively developed by Prior Labs and offers fast, zero-shot predictions, often outperforming tuned tree-based models and AutoML systems on suitable datasets.

Common errors

```
TypeError: 'TabPFNClassifier' object got an unexpected keyword argument 'N_ensemble_configurations'
```
cause The `N_ensemble_configurations` parameter was removed or changed in newer versions.

fix Remove the `N_ensemble_configurations` argument. Check the documentation for available parameters in your installed version. The default ensemble behavior is usually handled internally.
```
UserWarning: Running on CPU to estimate real data statistics. Note: performing inference on CPU is considerably slower than GPU. Consider calling 'set_device('cuda')' on your TabPFNClassifier to set TabPFN to GPU.
```
cause TabPFN detected that a GPU is available but is running on CPU by default or due to explicit 'cpu' device setting.

fix If you have a GPU, set `clf = TabPFNClassifier(device='cuda')` for significantly faster inference. Ensure PyTorch with CUDA support is installed.
```
RuntimeError: CUDA out of memory. Tried to allocate XXX MiB (GPU XXX; YYY MiB total capacity; ZZZ MiB already allocated; AAA MiB free; BBB MiB reserved in total by PyTorch)
```
cause The dataset (training and/or test set) is too large to fit into the GPU's VRAM.

fix Reduce the size of your training data (e.g., by sampling) or test data (by chunking `X_test` for prediction). Consider using a GPU with more VRAM, or switch to CPU inference (which will be slower).
```
TabPFNRegressor fails on constant input data
```
cause TabPFNRegressor might encounter errors when the target variable `y` in the training data is constant.

fix For truly constant targets, consider adding tiny, practically insignificant noise to the target variable `y` or implement explicit checks to skip fitting if `y` is constant and handle such cases separately (e.g., by predicting the constant value directly).
```
ImportError: cannot import name 'TabPFNClassifier' from 'tabpfn'
```
cause This usually indicates an incorrect installation, a Python version mismatch, or trying to import from an old or incorrect path.

fix Ensure `tabpfn` is installed with `pip install tabpfn`. Verify your Python environment meets the `>=3.9` requirement. If using a virtual environment, activate it before installing and importing.

Warnings

gotcha TabPFN is optimized for small to medium-sized datasets, typically up to 50,000 rows and 2,000 features. Performance significantly degrades on larger datasets, or if the number of classes exceeds common limits (e.g., 10 for the core model, though extensions exist).
Fix: For very large datasets, consider sampling, hybrid approaches (e.g., with Random Forests), or commercial versions. Use TabPFN Extensions for 'many_class' problems.
gotcha GPU is strongly recommended for TabPFN for optimal performance, even older ones with ~8GB VRAM. CPU inference is considerably slower and only feasible for very small datasets (≲1000 samples).
Fix: Ensure a CUDA-enabled PyTorch installation and set `device='cuda'` for `TabPFNClassifier` or `TabPFNRegressor`. If no GPU is available, consider using the TabPFN API Client for hosted inference.
breaking TabPFN requires Python 3.9 or newer due to reliance on modern language features. Using older Python versions will result in import errors or installation failures.
Fix: Upgrade your Python environment to 3.9, 3.10, or 3.11.
gotcha TabPFN performs internal data preprocessing (e.g., normalization, handling missing values, categorical features). Explicitly applying data scaling (e.g., StandardScaler) or one-hot encoding *before* feeding data to TabPFN is generally not recommended and can negatively impact performance.
Fix: Feed raw or minimally preprocessed numerical and categorical data directly to TabPFN. Let the model handle feature transformations.
gotcha Calling `predict` or `predict_proba` repeatedly for single test samples is highly inefficient, as each call recomputes the training set context.
Fix: Always use batch prediction mode by passing all test samples (`X_test`) in a single call to `clf.predict(X_test)` or `clf.predict_proba(X_test)`. If `X_test` is very large, split it into chunks (e.g., 1000 samples each) and process in batches.

Install

pip install tabpfn Stable release
pip install --upgrade tabpfn Upgrade to latest version

Imports

TabPFNClassifier
```
from tabpfn import TabPFNClassifier
```
TabPFNRegressor
```
from tabpfn import TabPFNRegressor
```

Quickstart

Demonstrates basic usage of TabPFNClassifier with a scikit-learn compatible interface for binary classification. For optimal performance, specify `device='cuda'` if a GPU is available. Note that the first execution may prompt a browser window for license acceptance.

import numpy as np
from tabpfn import TabPFNClassifier
from sklearn.model_selection import train_test_split

# Generate synthetic data
X = np.random.rand(100, 10) # 100 samples, 10 features
y = np.random.randint(0, 2, 100) # Binary classification target

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

# Initialize and use TabPFNClassifier (sklearn-like interface)
# The first call might trigger a license acceptance in browser.
clf = TabPFNClassifier(device='cpu') # Use 'cuda' if GPU is available
clf.fit(X_train, y_train)
predictions = clf.predict(X_test)
probabilities = clf.predict_proba(X_test)

print(f"Predictions: {predictions[:5]}")
print(f"Probabilities (first 5 samples):\n{probabilities[:5]}")

view raw JSON →