Search papers, labs, and topics across Lattice.
This paper investigates the scaling properties of Metropolised Markov Chain Monte Carlo (MCMC) algorithms as dimensionality increases, leveraging the symmetry inherent in the Metropolis-Hastings formula. The authors derive optimal scaling results for various proposal mechanisms, including implicit proposals and those based on differential equation integrators, and demonstrate how to construct gradient-based MALA-like proposals with reduced variance as dimensionality grows. Their findings not only encompass known results for established algorithms like Random Walk Metropolis and MALA but also extend to new scenarios involving products of distributions with varying scales.
Gradient-based MALA-like proposals can achieve variance scaling of \(O(1/d^\mu)\) with \(\mu\) arbitrarily small, challenging traditional scaling norms in high-dimensional MCMC.
We present a simple, yet general approach to study the scaling properties as the dimensionality of Metropolised MCMC sampling algorithms increases. The study relies ultimately on the symmetry of the Metropolis-Hastings formula. Our findings contain, as particular cases, many known results for the Random Walk Metropolis, MALA and other algorithms. In addition, they provide, in an easy way, new optimal scaling results for a variety of proposal mechanisms, including implicit proposals and proposals generated with the help of differential equation integrators. The analysis applies to targets that are products of a given, not necessarily univariate distribution, and also to cases where the different terms in the product are scaled differently. We show how to construct gradient-based MALA-like proposals where the variance of the proposal as the dimension $d$ increases may be taken as $O(1/d^渭)$, with $渭>0$ arbitrarily small, to be compared with the values $渭= 1$ for Random Walk Metropolis and $渭=1/3$ for MALA.
