Search papers, labs, and topics across Lattice.
This paper introduces FutureRTC, an innovative framework designed to enhance real-time Vision-Language-Action (VLA) policies by enabling asynchronous execution through anticipatory-conditioned action chunking. By incorporating a state correction module and an observation prediction module, FutureRTC effectively aligns predicted execution-time observations with actual states, addressing the critical issue of prediction-execution misalignment. Experimental results reveal that FutureRTC significantly improves robustness to inference delays, leading to smoother trajectories and higher task success rates in both simulated and real-world environments.
FutureRTC transforms real-time robot execution by seamlessly predicting execution-time observations, leading to smoother and faster task performance even under inference delays.
Real-time deployment of Vision-Language-Action (VLA) policies necessitates asynchronous execution, wherein subsequent action chunks are computed concurrently with the execution of the current chunk, leading to prediction-execution misalignment and manifesting as inter-chunk discontinuities. Existing methods either superficially smooth chunk boundaries, require costly policy optimization, or exclusively forward-predict proprioceptive states yet neglect critical visual observations. In this paper, we propose \textbf{FutureRTC}, a plug-and-play adaptation framework that predicts execution-time observations and states for asynchronous VLA control without modifying the underlying policy. Specifically, FutureRTC features a state correction module to compensate for the discrepancy between rolled-forward and actual execution-time proprioceptive states and an observation prediction module that forecasts execution-time visual representations by leveraging robot motion as an explicit physical prior through motion-aware feature transport and reconstruction. Furthermore, we introduce a policy consistency loss to align the action chunks generated from predicted contexts with those produced under the expected execution-time inputs of the VLA policy. Extensive experiments across simulated and real-world environments demonstrate that FutureRTC achieves superior robustness to inference delays, resulting in smoother trajectories, faster execution, and consistently higher task success rates.
