Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces a novel approach to Bayesian online learning by treating Bayesian update rules as individual experts and aggregating their predictions based on sequential predictive losses. This method allows for dynamic adaptation of learning rates and prior distributions, leading to improved performance in online conformal inference and Gaussian process regression. Key results demonstrate that the aggregate method competes effectively with the best expert in hindsight, achieving significant improvements in predictive accuracy and uncertainty quantification without the need for prior expert selection.
Aggregating Bayesian update rules as experts can outperform fixed strategies, achieving adaptive performance in online learning scenarios.
Bayesian online learning promises uncertainty-aware prediction on data streams, but its performance hinges on inferential choices, including learning rates, prior distributions and variational families, which are usually fixed before seeing the stream. We address this by treating Bayesian update rules as experts and aggregating the Bayesian experts according to sequential predictive losses. We prove that the resulting aggregate competes with the best expert in hindsight at an aggregation cost determined by how each expert's per-round performance is evaluated. We instantiate the framework in online conformal inference and Gaussian process regression. The conformal inference application yields a smoothed Bayesian counterpart of adaptive conformal inference with long-run randomized coverage, while the Gaussian process application gives an oracle inequality in cumulative predictive Kullback-Leibler risk and adaptation to unknown Hölder smoothness up to logarithmic factors. Experiments show that the aggregate tracks strong experts without oracle expert selection.