Search papers, labs, and topics across Lattice.
This paper introduces RoMAN-Flow, a novel offline reinforcement learning framework that enhances the practicality of Autoregressive Normalizing Flows (AR-NFs) for robotic manipulation by overcoming the sampling inefficiencies associated with policy optimization and deployment. By utilizing a sampling-free, advantage-weighted likelihood objective, RoMAN-Flow effectively prioritizes high-advantage actions from offline datasets, streamlining the policy optimization process. Experimental results show that RoMAN-Flow not only achieves competitive performance across various manipulation tasks but also significantly reduces inference latency, making it a viable solution for real-world applications.
RoMAN-Flow slashes inference latency while maintaining competitive performance, making AR-Normalizing Flows practical for real-world robotic manipulation.
Offline reinforcement learning improves robotic policies using previously collected data without further environment interaction. Yet prevalent diffusion- and flow-matching robot policies lack tractable likelihoods, limiting their use in likelihood-based offline RL post-training. AR-NFs offer both expressive action modeling and exact likelihood evaluation, but their sequential sampling incurs substantial sampling overhead during policy optimization and deployment. We present RoMAN-Flow (Robotic Manipulation with Autoregressive Normalizing Flows), an offline reinforcement learning framework that makes AR-NF policies practical for robotic manipulation by addressing this sampling bottleneck in both stages. During policy optimization, RoMAN-Flow employs a sampling-free, advantage-weighted likelihood objective that assigns higher likelihood to high-advantage actions from the offline dataset without sampling from the autoregressive policy. For efficient deployment, it distills the optimized autoregressive policy into a one-step action generator, enabling low-latency action prediction. Experiments across multiple simulated manipulation benchmarks and real-world robotic platforms demonstrate that RoMAN-Flow achieves competitive policy performance while substantially reducing inference latency. Code is available at https://github.com/konnyaku28/RoMAN-Flow.
