Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces Flow-Transformed Implicit Processes (FTIP), a novel variational inference method that enhances the expressiveness of posterior distributions in Bayesian function-space modeling. By employing normalizing flows instead of Gaussian distributions over combination weights, FTIP effectively captures complex posterior structures, including asymmetry and multimodality, which are often lost in traditional approaches. Experimental results demonstrate that FTIP significantly outperforms Gaussian-based methods in representing intricate function-space distributions, thereby improving the accuracy of posterior inference.
FTIP reveals that using normalizing flows can dramatically enhance the expressiveness of variational inference in function-space modeling, capturing complexities that Gaussian approximations miss.
Implicit-process priors define distributions over functions through flexible generative mechanisms, making them attractive for Bayesian function-space modelling. However, performing posterior inference with such priors is challenging because their induced function-space distributions are typically not available in closed form. One practical strategy is to approximate the prior using a finite collection of sampled functions, and then represent posterior functions as learned combinations of these samples. Existing approaches commonly place a Gaussian variational distribution over the combination weights. While tractable, this choice limits the shapes of posterior uncertainty that can be represented, especially when the true posterior is asymmetric, heavy-tailed, or multimodal. We propose Flow-Transformed Implicit Processes (FTIP), a variational inference method that makes this finite-dimensional function-space approximation more expressive. Instead of using a Gaussian distribution over the combination weights, FTIP uses a normalizing flow to define a richer variational distribution. This induces a flexible posterior distribution over functions while preserving tractable optimization. We train the model using a Black-Box α objective, allowing us to compare mass-covering and mode-seeking variational behaviour. Experiments show that FTIP captures asymmetric and multimodal posterior structure in function space that Gaussian coefficient approximations tend to smooth or collapse.