Search papers, labs, and topics across Lattice.
This paper investigates the last iterate of the stochastic subgradient method (SsGM) for one-dimensional convex Lipschitz objectives, establishing new bounds on optimization error. By employing fixed stepsizes of \(畏=螛(1/\sqrt n)\), the authors demonstrate that the last iterate achieves an error of order \(1/\sqrt n\) under additive i.i.d. subgradient noise, eliminating the previously established \((\log n)\) factor. Conversely, they reveal that without the i.i.d. assumption, the error can increase to \((\log n)/\sqrt n\), highlighting the limitations of the method under certain conditions and resolving an open question from prior research.
The last iterate of the stochastic subgradient method can achieve a significantly tighter optimization error bound, but only under strict noise assumptions.
We study the last iterate of the stochastic subgradient method for one-dimensional convex Lipschitz objectives. For a fixed horizon $n$, we consider the standard fixed stepsizes $畏=螛(1/\sqrt n)$. We prove that, for such stepsize policies, under additive i.i.d. subgradient noise with uniformly bounded variance, the last iterate features an optimization error of order $1/\sqrt n$, thereby removing the extra $(\log n)$ factor present in existing generic bounds. On the other hand, we show that without the i.i.d. assumption, the optimization error can be of order $(\log n)/\sqrt n$. Thus, under the uniformly bounded variance assumption alone, the last iterate of SsGM is suboptimal even in dimension one, resolving negatively an open problem posed in Koren and Segal, COLT, 2020.
