Search papers, labs, and topics across Lattice.
The authors formalize BrachistoneLR鈥攁 physics-inspired schedule mathematically equivalent to cosine annealing terminating at $E-1$ epochs鈥攁nd benchmark it against five standard policies across four architecture families and three datasets under strictly controlled conditions. This rigorous isolation resolves the confounding factors of optimizer, budget, and architecture changes that commonly plague empirical schedule comparisons. Smoothly decaying policies outperform constant and step-based decay by up to 2.5% on CIFAR-10, but the performance variance among smooth variants (BrachistoneLR, standard cosine, warmup-cosine) is statistically negligible at less than 0.06 accuracy points.
Despite the allure of physics-inspired curves, BrachistoneLR reduces algebraically to cosine annealing terminating one epoch early, proving that adopting smooth decay matters significantly while the choice between specific smooth parameterizations does not.
The learning-rate schedule is a consequential choice in training deep networks, yet the policies in common use are heuristic, and published comparisons are hard to read, because architecture, dataset, and budget tend to vary alongside the schedule. We study BrachistoneLR, a schedule built by mapping the vertical coordinate of the brachistochrone, the curve of fastest descent under gravity, onto the range between a peak and a floor rate. Expanding the definition shows it to be cosine annealing with the half-period set to E - 1 instead of E, the configuration a standard implementation gives when its period argument is one less than the number of epochs. The rate therefore reaches its floor at the last epoch trained rather than one epoch later, and we show this difference decays as E^-2, making it a short-horizon effect. We then benchmark six schedules over 72 runs on three image classification datasets (MNIST, Fashion-MNIST, CIFAR-10) and four architecture families (fully connected, convolutional, recurrent, residual), fixing the optimizer, data pipeline, and evaluation protocol so that only the schedule varies. Schedules that fall smoothly from peak to floor beat the constant rate and calendar-based decay by margins that grow with task difficulty, reaching 2.5 points of dataset mean on CIFAR-10. Within that leading group, BrachistoneLR, cosine annealing, and warmup-cosine lie within 0.06 accuracy points and 0.17 of a mean rank, which one seed per configuration cannot separate. BrachistoneLR is best on both residual networks and has the highest CIFAR-10 mean, and it sets no milestones, decay factor, warmup length, or restart period. We conclude that the shape of a schedule matters more than its parameterization, that the choice of whether to use a smooth schedule matters more than the choice among them, and that the terminal-rate distinction is worth attention only over short horizons.
