Search papers, labs, and topics across Lattice.
This paper introduces a continual learning framework for amortized Bayesian inference (ABI) to address performance degradation under model misspecification when faced with sequentially arriving, unlabeled real-world data. The framework decouples simulation-based pre-training from unsupervised sequential self-consistency (SC) fine-tuning. By incorporating episodic replay and elastic weight consolidation into the SC training, the proposed method mitigates catastrophic forgetting and improves posterior estimation accuracy compared to standard simulation-based training.
Continual self-consistency learning rescues amortized Bayesian inference from catastrophic forgetting, enabling robust posterior estimation even with sequentially arriving, real-world data.
Amortized Bayesian Inference (ABI) enables efficient posterior estimation using generative neural networks trained on simulated data, but often suffers from performance degradation under model misspecification. While self-consistency (SC) training on unlabeled empirical data can enhance network robustness, current approaches are limited to static, single-task settings and fail to handle sequentially arriving data or distribution shifts. We propose a continual learning framework for ABI that decouples simulation-based pre-training from unsupervised sequential SC fine-tuning on real-world data. To address the challenge of catastrophic forgetting, we introduce two adaptation strategies: (1) SC with episodic replay, utilizing a memory buffer of past observations, and (2) SC with elastic weight consolidation, which regularizes updates to preserve task-critical parameters. Across three diverse case studies, our methods significantly mitigate forgetting and yield posterior estimates that outperform standard simulation-based training, achieving estimates closer to MCMC reference, providing a viable path for trustworthy ABI across a range of different tasks.
