{"id":2118,"library":"microsoft-security-utilities-secret-masker","title":"Microsoft Security Utilities Secret Masker","description":"Microsoft Security Utilities - Secret Masker (version 1.0.0b4) is a Python library designed for the detection and masking of sensitive data. It provides built-in JSON-formatted detection rules, enabling users to identify and redact secrets using simple symbols or SHA256 hashes. This tool is part of Microsoft's internal security utilities and focuses on preventing secret exposure. It was last released on March 10, 2025, and is actively maintained in a beta state.","status":"active","version":"1.0.0b4","language":"en","source_language":"en","source_url":"https://github.com/microsoft/security-utilities","tags":["security","secrets","masking","utilities","microsoft","regex","pii"],"install":[{"cmd":"pip install microsoft-security-utilities-secret-masker","lang":"bash","label":"Install with pip"}],"dependencies":[],"imports":[{"note":"The primary class for secret detection and masking.","symbol":"SecretMasker","correct":"from microsoft_security_utilities_secret_masker import SecretMasker"},{"note":"Used to load predefined or custom regex patterns for secret detection.","symbol":"load_regex_patterns_from_json_file","correct":"from microsoft_security_utilities_secret_masker import load_regex_patterns_from_json_file"}],"quickstart":{"code":"from microsoft_security_utilities_secret_masker import SecretMasker, load_regex_patterns_from_json_file\n\n# Load built-in detection rules\nprecisely_classified_regex_patterns = load_regex_patterns_from_json_file('PreciselyClassifiedSecurityKeys.json')\nunclassified_regex_patterns = load_regex_patterns_from_json_file('UnclassifiedPotentialSecurityKeys.json')\n\n# Combine patterns\nregex_patterns = precisely_classified_regex_patterns.union(unclassified_regex_patterns)\n\n# Construct secret masker with chosen patterns\nsecret_masker = SecretMasker(regex_patterns)\n\n# Example usage\ninput_text = \"My API key is sk-1234567890abcdef1234567890abcdef and my email is test@example.com\"\n\ndetected_secrets = secret_masker.detect_secrets(input_text)\nprint(f\"Detected secrets: {detected_secrets}\")\n\nprocessed_input = secret_masker.mask_secrets(input_text)\nprint(f\"Masked input: {processed_input}\")\n\n# Example with custom masking character (e.g., SHA256)\nprocessed_input_sha256 = secret_masker.mask_secrets(input_text, mask_with_sha256=True)\nprint(f\"Masked with SHA256: {processed_input_sha256}\")","lang":"python","description":"This quickstart demonstrates how to initialize the SecretMasker with built-in detection rules, then use it to detect and mask secrets in an input string. It also shows an option for SHA256 hashing for masking."},"warnings":[{"fix":"Review changelogs and release notes carefully when updating. Pin to specific beta versions if stability is critical, and thoroughly test upgrades.","message":"The library is currently in a beta release (1.0.0b4). This means that API interfaces, behavior, and underlying implementations may change without strict adherence to semantic versioning until a stable 1.0.0 release. Users should be prepared for potential adjustments when upgrading to newer beta or release candidate versions.","severity":"gotcha","affected_versions":"1.0.0b4 and earlier beta versions"},{"fix":"If you require detailed detection information alongside the masked output, explicitly call `secret_masker.detect_secrets()` first, or ensure your library version includes the updated `mask_secrets` functionality (check for versions beyond 1.0.0b4 or consult the project's latest documentation). If not, chain the calls: `detections = secret_masker.detect_secrets(input_text); masked_output = secret_masker.mask_secrets(input_text)`.","message":"In earlier iterations, the `SecretMasker.mask_secrets` method might have only returned the masked string without providing details about the detected secrets that triggered the masking. While a 'detection callback' was noted as added in later development, users on version 1.0.0b4 (or older) should be aware of this.","severity":"gotcha","affected_versions":"1.0.0b4 and potentially earlier"},{"fix":"Regularly review and update the `PreciselyClassifiedSecurityKeys.json` and `UnclassifiedPotentialSecurityKeys.json` files from the official source, or provide your own custom regex patterns to `SecretMasker` to ensure comprehensive detection for your specific environment. Consider contributing new patterns back to the project if widely applicable.","message":"The effectiveness of secret detection relies heavily on the completeness and accuracy of the loaded JSON regex patterns. The built-in patterns provide a good starting point, but they may not cover all custom or evolving secret formats. Outdated or insufficient patterns can lead to undetected secrets (false negatives).","severity":"gotcha","affected_versions":"All versions"},{"fix":"Prioritize secure secret management solutions such as environment variables, Azure Key Vault, HashiCorp Vault, or other dedicated secret management services to store and retrieve credentials securely. Use this library as a secondary defense or for logging redaction, not as a replacement for secure secret storage.","message":"While this library helps mask secrets that appear in strings, the fundamental best practice for security is to avoid hardcoding secrets directly into code, configuration files, or committing them to version control systems. Relying solely on masking tools as a primary defense is a common cybersecurity mistake.","severity":"gotcha","affected_versions":"All versions (best practice guidance)"}],"env_vars":null,"last_verified":"2026-04-09T00:00:00.000Z","next_check":"2026-07-08T00:00:00.000Z"}