Track new results across pure and applied mathematics.
Search papers, labs, and topics across Lattice.
The authors present Bellman-gradient (BG) refinement, an amortized control framework that differentiates conditional rollouts through an existing actor within an identified Gaussian belief model to directly update an executable policy without requiring a learned value critic. Online feedback model-predictive control provides strong closed-loop robustness but incurs prohibitive runtime latency, while standard teacher-student policy distillation requires immense offline rollout compute. Across nonquadratic thrust, robotic arm, and docking benchmarks, BG attains performance within 0.13–0.53% of full feedback MPC at native 5–6 microsecond execution speeds while reducing policy construction compute by 9x to 60x compared to distillation baselines.
Real-time closed-loop policies can match expensive feedback MPC performance at 5-microsecond latencies while slashing training compute by up to 60x over standard distillation by differentiating rollouts directly through an existing actor.
Closed-loop planning accounts for future observation-dependent actions but can be expensive to repeat at deployment. Bellman-gradient (BG) refinement differentiates conditional rollouts through an existing actor, corrects the current action, and stores the result in an executable policy. A backward sweep reuses deployed future feedback; full-horizon rollouts in an identified Gaussian belief model need no learned value critic. A nonlinear error recursion links quadratic continuation error, local correction, and policy storage. A controlled non-LQG example exhibits second-order policy accuracy with consistent storage, while ideal affine LQG admits exact backward recovery. In nonquadratic thrust, BG reduces actor cost by 2.47-3.20% and remains within 0.13-0.53% of the tested feedback MPC; original policies execute in 5-6 microseconds in a seed-0 native audit. With matched storage, BG attains competitive plant costs at about one ninth (arm) and one sixtieth (docking) of feedback-teacher-plus-student construction time on one GPU, excluding shared learning and node preparation. Richer common maps substantially narrow some student-BG gaps. These results expose the roles of learned feedback, local correction, and storage in executable control.