Search papers, labs, and topics across Lattice.
This paper introduces FoMoVLA, a novel framework that enhances Vision-Language-Action (VLA) models by integrating explicit spatio-temporal supervision through joint learning of future feature foresight and sparse 2D point tracking. By combining these two approaches, FoMoVLA not only predicts future visual states but also provides a continuous motion path, significantly improving the model's ability to perform complex visuomotor tasks. Experimental results on multiple benchmarks, including LIBERO and RoboCasa GR-1 Tabletop, show that FoMoVLA achieves state-of-the-art performance and demonstrates strong zero-shot generalization capabilities.
Future feature foresight and sparse point tracking together can transform how VLA models navigate complex environments, leading to unprecedented performance in visuomotor tasks.
Vision-Language-Action (VLA) models have achieved impressive results in visuomotor policy learning, yet remain fundamentally reactive, mapping current observations and language to actions without explicit forward prediction of world dynamics. Existing visual foresight methods predict future visual states but lack explicit motion guidance: they show where to go but not how to get there. We argue that future feature prediction and sparse point tracking are naturally complementary: the former provides the goal state, while the latter captures the continuous motion path toward it. We propose FoMoVLA, a framework that augments VLA representations with explicit spatio-temporal supervision by jointly learning future feature foresight and sparse 2D point tracking, enhancing the continuous action policy. FoMoVLA introduces compact foresight tokens to decode future feature states, decodes sparse temporal 2D point trajectories to model compact geometric motion, and couples both through a lightweight future-conditioned cross-attention module that enables consistent reasoning between anticipated states and point dynamics. Extensive experiments on LIBERO, RoboCasa GR-1 Tabletop, and LIBERO-Plus demonstrate state-of-the-art performance and strong zero-shot generalization. Project page is available at https://liauto-research.github.io/FoMoVLA.
