Captum

Captum is an open-source model interpretability and understanding library for PyTorch. It provides a comprehensive suite of attribution algorithms to explain predictions of deep learning models, assess the importance of layers and neurons, and evaluate model robustness and concept influence. The library, currently at version 0.8.0, maintains an active development cycle with regular feature additions and improvements.