Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces a rate-transfer method that leverages finite-time Lyapunov estimates from an associated stochastic differential equation (SDE) to derive finite-time convergence bounds for stochastic gradient algorithms. The core technique simplifies convergence analysis by reducing it to a calculus-based verification of a Lyapunov generator inequality for the SDE. This approach successfully reproduces known convergence rates for standard stochastic gradient methods and offers a novel framework for analyzing new optimization algorithms via SDE calculus.
SDE Lyapunov estimates now directly translate to finite-time SGD convergence, simplifying analysis for novel optimizers.
We develop a rate-transfer method that converts finite-time Lyapunov estimates for an associated stochastic differential equation (SDE) into finite-time convergence bounds for stochastic gradient algorithms. Under generic moment and smoothness conditions, the convergence analysis of the stochastic gradient method reduces to a calculus argument verifying a Lyapunov generator inequality for the SDE. The method recovers established rates for classical stochastic gradient methods and provides a way to establish convergence rates for new optimization algorithms through calculus arguments applied to their associated SDEs.