Scrubadub: PII Redaction Library

Warnings

• breaking Version 2.0.0 introduced significant changes, including the splitting of the library into smaller sub-packages and a shift from loading all detectors by default to loading only a default set. Code relying on previously auto-loaded detectors (e.g., spaCy, Stanford NER) will need explicit `add_detector()` calls or installation of optional packages (`scrubadub_spacy`, `scrubadub_stanford`, `scrubadub_address`).
  Fix: Review the changelog for v2.0.0. For previously implicitly available detectors, explicitly install the relevant sub-package (e.g., `pip install scrubadub-spacy`) and add the detector to your `Scrubber` instance using `scrubber.add_detector(DetectorClass())`.
• breaking Python 2.7 and 3.5 support was dropped starting from version 2.0.0. If you require these Python versions, you must use `scrubadub` version 1.2.2 or earlier.
  Fix: Upgrade to Python 3.6+ or pin `scrubadub` to `==1.2.2` for older Python versions.
• gotcha Only a default set of detectors are loaded when initializing a `Scrubber` or using `scrubadub.clean()` since version 2.0.0. If you need functionality from optional or external detectors (e.g., `SpacyNameDetector`, `AddressDetector`), you must explicitly install their packages and add them to your `Scrubber` instance.
  Fix: Install the required optional packages (e.g., `pip install scrubadub-spacy`) and then use `scrubber.add_detector(DetectorClass())` to enable them.
• gotcha Attempting to add two detectors with the same name to a `Scrubber` instance will result in a `KeyError`.
  Fix: Ensure each detector you add has a unique name. If adding multiple instances of the same detector class with different configurations, assign a unique `name` parameter during instantiation (e.g., `EmailDetector(name='work_email_detector')`).
• gotcha Version 2.0.1 fixed an issue where the `scikit-learn` dependency was incorrectly named. Users might have encountered installation problems with `scrubadub==2.0.0` due to this.
  Fix: Ensure you are using `scrubadub==2.0.1` or later to avoid this specific dependency naming issue.

Install

• pip install scrubadub Core library
• pip install scrubadub scrubadub-spacy scrubadub-stanford scrubadub-address With optional detectors

Imports

• clean

  import scrubadub
  cleaned_text = scrubadub.clean(text)

  The primary convenience function for quick cleaning.
• Scrubber

  from scrubadub import Scrubber
  scrubber = Scrubber()
  cleaned_text = scrubber.clean(text)

  Use the Scrubber class for more fine-grained control, adding/removing specific detectors, or configuring post-processors.
• Detector

  from scrubadub.detectors import EmailDetector
  scrubber = Scrubber()
  scrubber.add_detector(EmailDetector())

  Import specific detector classes to add them to a Scrubber, especially for optional or external detectors.

Quickstart

This quickstart demonstrates the basic usage of `scrubadub.clean()` for straightforward PII redaction. It also illustrates how to use the `Scrubber` class to manually add detectors for more customized control over the scrubbing process, especially useful for optional or external detectors not loaded by default.

import scrubadub

text = "My cat can be contacted on example@example.com, or 1800 555-5555. His name is John Doe."
cleaned_text = scrubadub.clean(text)
print(cleaned_text)

# For more control, use the Scrubber class
from scrubadub import Scrubber
from scrubadub.detectors import TextBlobNameDetector # Example of an optional detector

scrubber = Scrubber()
# Add a detector if it's not enabled by default or for custom configuration
scrubber.add_detector(TextBlobNameDetector())
controlled_cleaned_text = scrubber.clean(text)
print(controlled_cleaned_text)