Search papers, labs, and topics across Lattice.
This paper introduces a high-order generator regression method for continuous-time policy evaluation using discrete trajectory data, addressing the limitations of first-order Bellman baselines. By estimating the time-dependent generator from multi-step transitions using moment-matching coefficients, the method achieves higher-order accuracy. The theoretical analysis provides an error decomposition and identifies conditions for observing gains from the higher-order approach, which is empirically validated through various benchmarks and ablation studies, demonstrating improved performance over the Bellman baseline.
Escaping the tyranny of Bellman's curse, a new method leverages multi-step transitions to achieve higher-order accuracy in continuous-time policy evaluation, outperforming traditional one-step recursion.
We study finite-horizon continuous-time policy evaluation from discrete closed-loop trajectories under time-inhomogeneous dynamics. The target value surface solves a backward parabolic equation, but the Bellman baseline obtained from one-step recursion is only first-order in the grid width. We estimate the time-dependent generator from multi-step transitions using moment-matching coefficients that cancel lower-order truncation terms, and combine the resulting surrogate with backward regression. The main theory gives an end-to-end decomposition into generator misspecification, projection error, pooling bias, finite-sample error, and start-up error, together with a decision-frequency regime map explaining when higher-order gains should be visible. Across calibration studies, four-scale benchmarks, feature and start-up ablations, and gain-mismatch stress tests, the second-order estimator consistently improves on the Bellman baseline and remains stable in the regime where the theory predicts visible gains. These results position high-order generator regression as an interpretable continuous-time policy-evaluation method with a clear operating region.
