SCAO is a sparse, second-order PyTorch optimizer designed as a high-throughput, drop-in replacement for AdamW.
Keras/TF implementation of AdamW, SGDW, NadamW, Warm Restarts, and Learning Rate multipliers
Nadir: Cutting-edge PyTorch optimizers for simplicity & composability! 🔥🚀💻
