XFormers

XFormers is a PyTorch-based library providing a collection of composable, optimized building blocks for Transformer models. It aims to accelerate deep learning research by offering flexible and highly efficient components, including advanced attention mechanisms and fused operations that often outperform native PyTorch implementations in terms of speed and memory usage. Actively developed by Meta Platforms, Inc., the library frequently releases updates, with the current stable version being 0.0.35.