Search papers, labs, and topics across Lattice.
This paper introduces Momentum SVGD-EM, an accelerated version of the SVGD-EM algorithm for maximum marginal likelihood estimation (MMLE). The acceleration is achieved by incorporating Nesterov momentum both in the parameter updates and in the space of probability measures within the EM framework. Experiments across various tasks demonstrate that Momentum SVGD-EM consistently accelerates convergence compared to standard SVGD-EM, in both low- and high-dimensional settings.
Nesterov momentum can significantly speed up SVGD-EM, a particle-based algorithm for maximum marginal likelihood estimation.
Maximum marginal likelihood estimation (MMLE) can be formulated as the optimization of a free energy functional. From this viewpoint, the Expectation-Maximisation (EM) algorithm admits a natural interpretation as a coordinate descent method over the joint space of model parameters and probability measures. Recently, a significant body of work has adopted this perspective, leading to interacting particle algorithms for MMLE. In this paper, we propose an accelerated version of one such procedure, based on Stein variational gradient descent (SVGD), by introducing Nesterov acceleration in both the parameter updates and in the space of probability measures. The resulting method, termed Momentum SVGD-EM, consistently accelerates convergence in terms of required iterations across various tasks of increasing difficulty, demonstrating effectiveness in both low- and high-dimensional settings.
