Learning Rate Warmup in PyTorch
A comprehensive, research-driven collection of learning rate schedulers for PyTorch — 17 schedulers, composable warmup, opinionated presets, and first-class paper references.
A Warmup Scheduler for Pytorch to achieve the warmup learning rate at the beginning of training.
