Search papers, labs, and topics across Lattice.
This paper introduces the Ensemble Controlled-flow Filter (EnCF), a novel approach to implicit data assimilation that addresses challenges posed by many-to-one and non-smooth observation mechanisms. By defining the analysis law as an energy tilt of the forecast distribution and employing stochastic controlled flow, EnCF effectively learns observation-dependent control through adjoint matching of terminal energy gradients. The results demonstrate that while traditional Kalman-type filters excel with smooth additive-Gaussian observations, EnCF outperforms them in scenarios involving non-Gaussian, multimodal, and implicit observation models.
EnCF outperforms traditional filters in complex observation scenarios, revealing a new frontier in data assimilation techniques.
Data assimilation estimates the state of a dynamical system from model forecasts and incoming observations. Many observation mechanisms, however, are many-to-one, implicit, non-smooth, or accessible only through simulation, and need not provide the residual structures or likelihood guidance required by existing ensemble filters. We introduce implicit data assimilation, in which the analysis law is defined as an energy tilt of the forecast distribution. We then propose the Ensemble Controlled-flow Filter (EnCF), which realizes this update through a stochastic controlled flow and learns the observation-dependent control by adjoint matching from terminal energy gradients. For simulator-defined observations, EnCF-LF learns a surrogate conditional energy from samples and applies the same controlled-flow solver. We prove ideal exactness, derive a one-step error decomposition, and establish non-accumulation of local errors under filter stability. Numerical results show that Kalman-type filters remain preferable for smooth additive-Gaussian observations, while the proposed methods are better suited to non-Gaussian, many-to-one, multimodal, and implicit observation models.
NUS