SMDebug RulesConfig

raw JSON →
1.0.1 verified Tue May 12 auth: no python install: stale

SMDebug RulesConfig is a Python library that provides a mapping of built-in rules with default configurations for Amazon SageMaker Debugger. It helps users specify these rules and common collection configurations without needing to handle low-level details, working in conjunction with the Amazon SageMaker Python SDK. The current version is 1.0.1, released in December 2020. While the package itself has not seen recent updates, its functionality remains an integral part of the SageMaker Debugger ecosystem.

pip install smdebug-rulesconfig
error ModuleNotFoundError: No module named 'smdebug_rulesconfig'
cause The 'smdebug-rulesconfig' Python package is not installed in the current environment or is not accessible in the Python path.
fix
Install the package using pip: pip install smdebug-rulesconfig
error ImportError: cannot import name 'LossNotDecreasing' from 'smdebug_rulesconfig'
cause Specific rules like `LossNotDecreasing` are located within the `smdebug_rulesconfig.rules` submodule, not directly under the top-level `smdebug_rulesconfig` package.
fix
Import the rule from the correct submodule: from smdebug_rulesconfig.rules import LossNotDecreasing
error ModuleNotFoundError: No module named 'smdebug_rulesconfig.debugger_rules'
cause The `smdebug-rulesconfig` library consolidated rule definitions under the `smdebug_rulesconfig.rules` submodule. Older examples or documentation might reference the deprecated `debugger_rules` submodule.
fix
Update the import statement to use the current rules submodule: from smdebug_rulesconfig.rules import Overfit (replace Overfit with the desired rule name).
error TypeError: Parameter rule_configuration must be a dictionary. Got <class 'smdebug_rulesconfig.rules.loss_not_decreasing.LossNotDecreasing'> instead.
cause The `sagemaker.debugger.Rule` constructor expects the `rule_configuration` parameter to be a dictionary, but a `smdebug-rulesconfig` rule class (or instance) was provided directly without calling its `.base_config()` method.
fix
Call the .base_config() method on the rule class to obtain the required dictionary configuration: rules = [LossNotDecreasing.base_config()]
breaking Major version 1.0.0 introduced the ability to specify profiler rules, and 1.0.1 added action classes for rules (e.g., `StopTraining`, `Email`). While older rule definitions might still work, new features and potentially updated underlying behavior for rule processing necessitate awareness of these version changes when upgrading or using new capabilities.
fix Review SageMaker Python SDK and `smdebug` documentation for best practices when upgrading, especially concerning profiler rules and rule actions. Ensure your `sagemaker` and `smdebug` client libraries are also up-to-date.
gotcha This library (smdebug-rulesconfig) is tightly integrated with the Amazon SageMaker Python SDK and the `smdebug` client library. To utilize the latest Debugger features and ensure compatibility, it is crucial to keep both `sagemaker` and `smdebug` packages updated in your environment, not just `smdebug-rulesconfig`.
fix Always run `pip install -U sagemaker smdebug` alongside `pip install -U smdebug-rulesconfig` to ensure all components are at compatible and recent versions.
gotcha The SageMaker DebuggerHookConfig is initialized by default for framework estimators (e.g., TensorFlow, PyTorch) to minimize code changes for debugging. If you do not intend to use Debugger, you must explicitly disable the hook.
fix Set `debugger_hook_config=False` when initializing your SageMaker framework estimator, e.g., `estimator = TensorFlow(..., debugger_hook_config=False)`.
gotcha The `smdebug-rulesconfig` library focuses on *configuring* rules. The actual execution and analysis of these rules occur within the broader Amazon SageMaker Debugger service, often leveraging the `smdebug` client library for data retrieval and custom rule logic. Confusing these roles can lead to unexpected behavior if not properly integrated into a SageMaker training job.
fix Understand that `smdebug-rulesconfig` provides the rule definitions, which are then passed to a SageMaker Estimator. SageMaker handles running these rules. For local or custom analysis, the `smdebug` client library is used to read debug data. Refer to the SageMaker Debugger developer guide for a comprehensive understanding of the workflow.
python os / libc status wheel install import disk mem side effects
3.10 alpine (musl) wheel - - 18.0M - broken
3.10 alpine (musl) - - - - - -
3.10 slim (glibc) wheel 1.6s - 19M - broken
3.10 slim (glibc) - - - - - -
3.11 alpine (musl) wheel - - 19.9M - broken
3.11 alpine (musl) - - - - - -
3.11 slim (glibc) wheel 1.6s - 20M - broken
3.11 slim (glibc) - - - - - -
3.12 alpine (musl) wheel - - 11.7M - broken
3.12 alpine (musl) - - - - - -
3.12 slim (glibc) wheel 1.4s - 12M - broken
3.12 slim (glibc) - - - - - -
3.13 alpine (musl) wheel - - 11.5M - broken
3.13 alpine (musl) - - - - - -
3.13 slim (glibc) wheel 1.4s - 12M - broken
3.13 slim (glibc) - - - - - -
3.9 alpine (musl) wheel - - 17.5M - broken
3.9 alpine (musl) - - - - - -
3.9 slim (glibc) wheel 1.7s - 18M - broken
3.9 slim (glibc) - - - - - -

This quickstart demonstrates how to instantiate built-in SageMaker Debugger rules using `smdebug-rulesconfig` via the `sagemaker.debugger` module. It shows both a basic rule configuration and a more advanced example with custom parameters and tensor collection specifications, which are typically passed to a SageMaker Estimator.

from sagemaker.debugger import Rule, CollectionConfig, rule_configs
from sagemaker.estimator import Estimator # Placeholder for a SageMaker Estimator
import os

# In a real scenario, replace this with your actual SageMaker Estimator setup
# and ensure AWS credentials are configured (e.g., via environment variables or AWS CLI)
# For example: estimator = TensorFlow(role=os.environ.get('SAGEMAKER_ROLE', 'arn:aws:iam::123456789012:role/SageMakerRole'), ...)
# We use a placeholder here for quickstart reproducibility.

# Example: Vanilla built-in rule without customization
# This would typically be passed to a SageMaker Estimator's 'rules' parameter.
rule_vanishing_gradient = Rule.sagemaker(rule_configs.vanishing_gradient())
print(f"Vanishing Gradient Rule: {rule_vanishing_gradient.name}")

# Example: Built-in rule with customization
# This demonstrates how to customize a rule's parameters and collections.
rule_customized_weight_update = Rule.sagemaker(
    base_config=rule_configs.weight_update_ratio(),
    name="my_custom_wup_rule", # Optional
    # container_local_path="/local/path", # Optional, if running locally or specific container path
    # s3_output_path="s3://your-s3-bucket/debug-output/", # Optional, overrides default
    rule_parameters={
        "threshold": "0.001" # Example parameter customization
    },
    collections_to_save=[
        CollectionConfig(
            name="weights", # Required. Debugger will collect tensors for this collection.
            parameters={
                "save_interval": "100" # Example collection parameter
            }
        )
    ]
)
print(f"Customized Weight Update Ratio Rule: {rule_customized_weight_update.name}")
print(f"  Rule parameters: {rule_customized_weight_update.rule_parameters}")
print(f"  Collections to save: {[c.name for c in rule_customized_weight_update.collections_to_save]}")