LIME: Local Interpretable Model-Agnostic Explanations

Warnings

• breaking Python 2 support was dropped in version 0.2.0.0. LIME now requires Python 3.5 or newer.
  Fix: Ensure your project uses Python 3.5 or a later version. Upgrade your Python environment if necessary.
• gotcha For `LimeTabularExplainer`, `training_data` is crucial. It is used to compute statistics (mean, std dev, frequencies) for feature perturbation and discretization. Providing a non-representative or empty `training_data` can lead to inaccurate or misleading explanations.
  Fix: Always provide `LimeTabularExplainer` with a representative sample of your training data (as a NumPy array) to ensure accurate feature statistics are calculated.
• gotcha The `predict_fn` passed to `explain_instance` must return probabilities for classification tasks (e.g., `model.predict_proba`) and raw predicted values for regression tasks (e.g., `model.predict`). Mismatching this can cause errors or incorrect explanations.
  Fix: Verify that your `predict_fn` matches the expected output type for LIME based on your model's task (probabilities for classification, raw values for regression).
• gotcha LIME explanations can be sensitive to hyperparameters like `num_samples` (number of perturbed samples) and the choice of distance metric, potentially leading to varied or unstable explanations across runs or slight changes in settings.
  Fix: Experiment with different hyperparameters to assess the stability of explanations. Consider the theoretical limitations and potential for local instability, especially in high-dimensional or complex feature spaces.

Install

• pip install lime Install stable version

Imports

• LimeTabularExplainer

  from lime.lime_tabular import LimeTabularExplainer

  Explainer classes are located within sub-modules (e.g., lime_tabular, lime_text, lime_image), not directly under the top-level 'lime' package.
• LimeTextExplainer

  from lime.lime_text import LimeTextExplainer

  Explainer classes are located within sub-modules (e.g., lime_tabular, lime_text, lime_image), not directly under the top-level 'lime' package.
• LimeImageExplainer

  from lime.lime_image import LimeImageExplainer

  Explainer classes are located within sub-modules (e.g., lime_tabular, lime_text, lime_image), not directly under the top-level 'lime' package.

Quickstart

This quickstart demonstrates how to use `LimeTabularExplainer` to explain an individual prediction from a scikit-learn RandomForestClassifier on the Iris dataset. It covers data preparation, model training, explainer initialization, and generating/displaying the explanation.

import numpy as np
import sklearn
import sklearn.ensemble
from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split

import lime
import lime.lime_tabular

# 1. Prepare Data and Model
iris = load_iris()
X = iris.data
y = iris.target
feature_names = iris.feature_names
class_names = iris.target_names

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

model = sklearn.ensemble.RandomForestClassifier(n_estimators=100, random_state=42)
model.fit(X_train, y_train)

# 2. Create a LIME Explainer for Tabular Data
explainer = lime.lime_tabular.LimeTabularExplainer(
    training_data=X_train, 
    feature_names=feature_names, 
    class_names=class_names, 
    mode='classification'
)

# 3. Choose an instance to explain
i = np.random.randint(0, X_test.shape[0])
instance_to_explain = X_test[i]

# 4. Generate the explanation
explanation = explainer.explain_instance(
    data_row=instance_to_explain, 
    predict_fn=model.predict_proba, 
    num_features=2
)

# 5. Print the explanation
print(f"Explaining instance: {instance_to_explain}")
print(f"Predicted class: {iris.target_names[model.predict(instance_to_explain.reshape(1, -1))[0]]}")
print("--- Local Explanation ---")
print(explanation.as_list())

# For visualization in a Jupyter Notebook:
# explanation.show_in_notebook(show_table=True)