Yuhan Ye

Vanilla gradient descent cannot beat the silver stepsize schedule, proving that predetermined learning rates hit a hard theoretical limit of $\mathcal{O}(n^{-\log_2(1+\sqrt{2})})$ without momentum.

Achieving a new anytime lower bound of \(Ω(n^{-1.2408})\) reveals critical insights into the acceleration limits of gradient descent.