Search papers, labs, and topics across Lattice.
This paper introduces new formulations of Hamiltonian Monte Carlo (HMC) tailored for lazy, infinite-dimensional probabilistic programs, leveraging Haskell's lazy evaluation capabilities. By analyzing the gradients of likelihood functions through a novel framework called PACAP, the authors demonstrate that these infinite-dimensional models can still yield finitely supported gradients, enabling effective inference. The proposed HMC variants and No-U-Turn Sampler are validated through experiments on Gaussian mixture clustering, random walks, and piecewise-constant regression, showcasing their practical utility in complex probabilistic settings.
Infinite-dimensional probabilistic models can now be efficiently sampled using Hamiltonian Monte Carlo without losing the benefits of lazy evaluation.
Hamiltonian Monte Carlo (HMC) is a successful generic inference method in probabilistic programming, but in its ordinary formulation it needs gradients and finite-dimensional parameter spaces. In Haskell, lazy evaluation lets probabilistic programs express stochastic processes and other non-parametric Bayesian models over implicit infinite-dimensional spaces. This paper develops new formulations of gradient-based HMC for this infinite-dimensional setting, via lazy evaluation. For automatic differentiation, we provide an analysis based on a new notion of "piecewise analytic under cylindrical analytic partition" (PACAP), to show that even if a program is infinite-dimensional and defined lazily, the gradient of the likelihood function is finitely supported. For the Monte Carlo method itself, we develop several HMC variants and a No-U-Turn Sampler that operate over the infinite-dimensional parameter space but are still productive because of lazy evaluation. Experiments cover Gaussian mixture clustering, random walks, and piecewise-constant regression with Poisson-process changepoints.
