Search papers, labs, and topics across Lattice.
This paper introduces Latent Action Flow Policy (LAFP), a novel approach that utilizes flow matching to preserve latent action structures in latent policy learning from large-scale unlabeled video data. By addressing the misalignment between latent and physical actions during action decoder training through an inference-time interpolation mechanism, LAFP overcomes the limitations of traditional behavior cloning methods that reduce multimodal action distributions. Experimental results show that LAFP achieves a 10-15% improvement in success rates on downstream imitation learning tasks with minimal additional inference overhead.
LAFP achieves up to 15% higher success rates in imitation learning by preserving the multimodal structure of latent actions, challenging the limitations of traditional behavior cloning.
Learning high-quality latent actions from large-scale unlabeled videos, coupled with limited real-world interaction data for training an action decoder, has emerged as a promising paradigm for scalable latent policy learning. However, existing approaches typically rely on behavior cloning, which tends to collapse inherently multimodal action distributions into unimodal ones, thereby degrading the pretrained latent action structure. While flow matching provides a potential alternative, directly applying it leads to a misalignment between latent actions and physical actions during action decoder training, due to the stochastic nature of the learned policy. To address these, we propose Latent Action Flow Policy (LAFP), which leverages flow matching for latent policy learning and introduces an inference-time interpolation mechanism to mitigate stochasticity-induced misalignment. Experimental results demonstrate that LAFP consistently outperforms prior methods on downstream imitation learning tasks, achieving up to 10-15% improvement in success rate while incurring less than 1x additional inference overhead.
