Search papers, labs, and topics across Lattice.
The paper introduces Field Least-Energy Actor-Critic (FLAC), a likelihood-free maximum entropy RL framework for continuous control with iterative generative policies. FLAC formulates policy optimization as a Generalized Schrödinger Bridge (GSB) problem, using kinetic energy of the velocity field as a proxy for divergence from a high-entropy reference process, thus avoiding explicit action density estimation. Experiments on high-dimensional benchmarks demonstrate that FLAC achieves superior or comparable performance to existing methods.
Ditch density estimation in maximum entropy RL: FLAC leverages kinetic energy regularization within a Schrödinger Bridge framework to achieve state-of-the-art performance without ever needing action log-densities.
Iterative generative policies, such as diffusion models and flow matching, offer superior expressivity for continuous control but complicate Maximum Entropy Reinforcement Learning because their action log-densities are not directly accessible. To address this, we propose Field Least-Energy Actor-Critic (FLAC), a likelihood-free framework that regulates policy stochasticity by penalizing the kinetic energy of the velocity field. Our key insight is to formulate policy optimization as a Generalized Schrödinger Bridge (GSB) problem relative to a high-entropy reference process (e.g., uniform). Under this view, the maximum-entropy principle emerges naturally as staying close to a high-entropy reference while optimizing return, without requiring explicit action densities. In this framework, kinetic energy serves as a physically grounded proxy for divergence from the reference: minimizing path-space energy bounds the deviation of the induced terminal action distribution. Building on this view, we derive an energy-regularized policy iteration scheme and a practical off-policy algorithm that automatically tunes the kinetic energy via a Lagrangian dual mechanism. Empirically, FLAC achieves superior or comparable performance on high-dimensional benchmarks relative to strong baselines, while avoiding explicit density estimation.
