What-If Tool TensorBoard Plugin

1.8.1 · maintenance · verified Fri Apr 10

The tensorboard-plugin-wit package integrates the What-If Tool (WIT) into TensorBoard, providing an interactive visual interface for understanding black-box machine learning models. It enables users to perform inference on datasets, visualize results, and manually or programmatically edit examples to observe how changes affect model predictions. The current version is 1.8.1, released on January 5, 2022. The underlying What-If Tool project is hosted on GitHub under `pair-code/what-if-tool` but is no longer actively maintained.

Warnings

deprecated The What-If Tool and its TensorBoard plugin are no longer actively maintained. Users are strongly encouraged to consider the actively maintained Learning Interpretability Tool (LIT) as an alternative for model understanding and interpretability.
Fix: Migrate to the Learning Interpretability Tool (LIT) for ongoing support and new features.
gotcha Full functionality of the What-If Tool in TensorBoard requires a model served via TensorFlow Serving. The model must expose its prediction interface through TensorFlow Serving's classify, regress, or predict API.
Fix: Ensure your machine learning model is deployed using TensorFlow Serving before attempting to analyze it with the What-If Tool in TensorBoard. Alternatively, use 'notebook mode' or a custom Python prediction function provided to TensorBoard with `--whatif-use-unsafe-custom-prediction`.
gotcha Input datasets for analysis within the What-If Tool in TensorBoard must be in TFRecord file format containing TensorFlow 'Example' protocol buffers.
Fix: Convert your dataset into the TFRecord format with TensorFlow Example protos. The TFRecord file needs to be accessible by the TensorBoard web server, typically under the `--logdir` or a path specified by `--whatif-data-dir`.
breaking Mixing TensorFlow 1.x and 2.x environments or incompatible TensorBoard versions can lead to issues such as 'duplicate plugins' or 'ImportError: TensorBoard logging requires TensorBoard version 1.15 or above'. TensorBoard versions 2.4.0 and above explicitly require TensorFlow 2.x.
Fix: Ensure consistent versions of TensorFlow and TensorBoard. If using TF 2.x, ensure TensorBoard is also 2.x compatible (preferably the latest compatible version). Consider using isolated virtual environments (e.g., `venv` or `conda`) to manage dependencies.

Install

pip install tensorboard-plugin-wit Install with pip

Imports

What-If Tool Plugin
```
The plugin is primarily accessed via the TensorBoard UI after launching TensorBoard. There is no direct Python import for the main plugin functionality, though parts of the underlying 'witwidget' library might be imported for advanced programmatic use.
```
The 'tensorboard-plugin-wit' package registers the What-If Tool as a TensorBoard plugin. Users interact with it through the TensorBoard web interface after launching TensorBoard. Direct programmatic control over the UI from Python is not the standard usage pattern.

Quickstart

This quickstart demonstrates how to generate basic TensorBoard logs. After running this Python code, launch TensorBoard from your terminal, pointing it to the parent directory of your logs. Once TensorBoard is running, navigate to its web interface (usually http://localhost:6006) and select the 'What-If Tool' from the dashboard dropdown. Note that for the What-If Tool to be fully functional, you would typically need a model served via TensorFlow Serving and a dataset in TFRecord format. This example primarily shows how to set up a basic TensorBoard environment where the plugin can be accessed.

import tensorflow as tf
import numpy as np
import os
from datetime import datetime

log_dir = os.path.join("logs", "whatif_example_" + datetime.now().strftime("%Y%m%d-%H%M%S"))
file_writer = tf.summary.create_file_writer(log_dir)

# Create some dummy data to log to TensorBoard
with file_writer.as_default():
    tf.summary.scalar('my_metric/loss', 0.5, step=0)
    tf.summary.scalar('my_metric/accuracy', 0.8, step=0)
    tf.summary.text('my_experiment/details', 'This is a test run for What-If Tool integration.', step=0)
    # What-If Tool also requires a TFRecord file with examples and a served model
    # This example only shows basic TensorBoard logging to demonstrate a runnable context.
    # For full WIT functionality, prepare TFRecord data and serve a model with TensorFlow Serving.

print(f"TensorBoard logs written to: {log_dir}")
print("To launch TensorBoard, run in your terminal:")
print(f"tensorboard --logdir {os.path.dirname(log_dir)}")
print("Then navigate to http://localhost:6006 and look for the What-If Tool dashboard.")

view raw JSON →