Microsoft Security Utilities Secret Masker

Warnings

• gotcha The library is currently in a beta release (1.0.0b4). This means that API interfaces, behavior, and underlying implementations may change without strict adherence to semantic versioning until a stable 1.0.0 release. Users should be prepared for potential adjustments when upgrading to newer beta or release candidate versions.
  Fix: Review changelogs and release notes carefully when updating. Pin to specific beta versions if stability is critical, and thoroughly test upgrades.
• gotcha In earlier iterations, the `SecretMasker.mask_secrets` method might have only returned the masked string without providing details about the detected secrets that triggered the masking. While a 'detection callback' was noted as added in later development, users on version 1.0.0b4 (or older) should be aware of this.
  Fix: If you require detailed detection information alongside the masked output, explicitly call `secret_masker.detect_secrets()` first, or ensure your library version includes the updated `mask_secrets` functionality (check for versions beyond 1.0.0b4 or consult the project's latest documentation). If not, chain the calls: `detections = secret_masker.detect_secrets(input_text); masked_output = secret_masker.mask_secrets(input_text)`.
• gotcha The effectiveness of secret detection relies heavily on the completeness and accuracy of the loaded JSON regex patterns. The built-in patterns provide a good starting point, but they may not cover all custom or evolving secret formats. Outdated or insufficient patterns can lead to undetected secrets (false negatives).
  Fix: Regularly review and update the `PreciselyClassifiedSecurityKeys.json` and `UnclassifiedPotentialSecurityKeys.json` files from the official source, or provide your own custom regex patterns to `SecretMasker` to ensure comprehensive detection for your specific environment. Consider contributing new patterns back to the project if widely applicable.
• gotcha While this library helps mask secrets that appear in strings, the fundamental best practice for security is to avoid hardcoding secrets directly into code, configuration files, or committing them to version control systems. Relying solely on masking tools as a primary defense is a common cybersecurity mistake.
  Fix: Prioritize secure secret management solutions such as environment variables, Azure Key Vault, HashiCorp Vault, or other dedicated secret management services to store and retrieve credentials securely. Use this library as a secondary defense or for logging redaction, not as a replacement for secure secret storage.

Install

• pip install microsoft-security-utilities-secret-masker Install with pip

Imports

• SecretMasker

  from microsoft_security_utilities_secret_masker import SecretMasker

  The primary class for secret detection and masking.
• load_regex_patterns_from_json_file

  from microsoft_security_utilities_secret_masker import load_regex_patterns_from_json_file

  Used to load predefined or custom regex patterns for secret detection.

Quickstart

This quickstart demonstrates how to initialize the SecretMasker with built-in detection rules, then use it to detect and mask secrets in an input string. It also shows an option for SHA256 hashing for masking.

from microsoft_security_utilities_secret_masker import SecretMasker, load_regex_patterns_from_json_file

# Load built-in detection rules
precisely_classified_regex_patterns = load_regex_patterns_from_json_file('PreciselyClassifiedSecurityKeys.json')
unclassified_regex_patterns = load_regex_patterns_from_json_file('UnclassifiedPotentialSecurityKeys.json')

# Combine patterns
regex_patterns = precisely_classified_regex_patterns.union(unclassified_regex_patterns)

# Construct secret masker with chosen patterns
secret_masker = SecretMasker(regex_patterns)

# Example usage
input_text = "My API key is sk-1234567890abcdef1234567890abcdef and my email is test@example.com"

detected_secrets = secret_masker.detect_secrets(input_text)
print(f"Detected secrets: {detected_secrets}")

processed_input = secret_masker.mask_secrets(input_text)
print(f"Masked input: {processed_input}")

# Example with custom masking character (e.g., SHA256)
processed_input_sha256 = secret_masker.mask_secrets(input_text, mask_with_sha256=True)
print(f"Masked with SHA256: {processed_input_sha256}")