HeavyBall: Compile-first PyTorch optimizer library

JSON →
library 3.0.0 ·python
verified May 26, 2026

HeavyBall is a PyTorch optimizer library that emphasizes 'compile-first' design, assembling optimizers from composable, compiled building blocks. It provides API-compatible replacements for `torch.optim` optimizers like AdamW, SGD, and RMSprop, along with over 30 specialized optimizers such as Muon, SOAP/Shampoo, PSGD, and Schedule-Free. Currently at version 3.0.0, the library is actively maintained with a focus on `torch.compile` fusion, Triton kernel optimization, and memory efficiency, including features like ECC state compression.

total hits 15
actors 6 distinct systems
last hit 3d ago MetaBot
MetaBot
4
GPTBot
2
Script
1
Search engines
1
Humans
2

top countries 🇺🇸 United States · 🇨🇦 Canada · 🇳🇴 Norway · 🇩🇪 Germany