Search papers, labs, and topics across Lattice.
This paper establishes a functional central limit theorem (FCLT) for stochastic gradient descent (SGD) applied to convex optimization problems under mild regularity conditions. The FCLT characterizes the algorithm's trajectory fluctuations around the minimizer by identifying a diffusion limit. This result extends beyond classical central limit theorems for the last iterate or Polyak-Ruppert averaging by capturing the temporal structure of fluctuations, and applies to non-smooth settings like robust location estimation.
SGD's entire path, not just its final destination, is now characterized by a functional CLT, revealing its fluctuations as a diffusion process even in non-smooth settings.
We study the asymptotic shape of the trajectory of the stochastic gradient descent algorithm applied to a convex objective function. Under mild regularity assumptions, we prove a functional central limit theorem for the properly rescaled trajectory. Our result characterizes the long-term fluctuations of the algorithm around the minimizer by providing a diffusion limit for the trajectory. In contrast with classical central limit theorems for the last iterate or Polyak-Ruppert averages, this functional result captures the temporal structure of the fluctuations and applies to non-smooth settings such as robust location estimation, including the geometric median.
