Search papers, labs, and topics across Lattice.
This paper generalizes the Mirror Descent (MD) algorithm to Riemannian manifolds, creating a Riemannian Mirror Descent (RMD) framework. The authors develop both deterministic and stochastic variants of RMD, providing non-asymptotic convergence guarantees for each. They demonstrate that RMD recovers Curvilinear Gradient Descent (CGD) on the Stiefel manifold, and the stochastic RMD yields a stochastic extension of CGD, useful for large-scale manifold optimization.
Mirror Descent, a workhorse of large-scale optimization, now has a Riemannian generalization with convergence guarantees, opening doors to efficient optimization on curved spaces.
Mirror Descent (MD) is a scalable first-order method widely used in large-scale optimization, with applications in image processing, policy optimization, and neural network training. This paper generalizes MD to optimization on Riemannian manifolds. In particular, we develop a Riemannian Mirror Descent (RMD) framework via reparameterization and further propose a stochastic variant of RMD. We also establish non-asymptotic convergence guarantees for both RMD and stochastic RMD. As an application to the Stiefel manifold, our RMD framework reduces to the Curvilinear Gradient Descent (CGD) method proposed in [26]. Moreover, when specializing the stochastic RMD framework to the Stiefel setting, we obtain a stochastic extension of CGD, which effectively addresses large-scale manifold optimization problems.
