Track manipulation, locomotion and embodied systems.
Search papers, labs, and topics across Lattice.
This paper investigates whether reallocating training budget from action prediction to auxiliary objectives like latent-action (LA) and future-vision (FV) prediction can improve Vision-Language-Action (VLA) policy learning. By introducing a unified framework with a sample-level router, the authors demonstrate that suitable LA and FV supervision consistently outperforms action-only training, even when action prediction constitutes less than half the training budget. Diagnostic analyses reveal that FV targets better capture absolute position and task identity, suggesting that the choice and allocation of prediction targets are critical design decisions.
Auxiliary future-vision prediction targets, not just action prediction, unlock superior robot policy performance even when action supervision is halved.
Vision-language-action (VLA) policies are typically trained with action prediction as the primary supervision signal, since robots ultimately execute actions. Auxiliary objectives such as latent-action (LA) and future-vision (FV) prediction are often added to improve policy learning, yet they compete with action prediction for the same training budget. We ask whether, under a fixed budget, allocating a substantial share of training samples to such objectives, rather than to further action prediction, can improve policy learning. We build a unified VLA framework that couples a two-stream multimodal diffusion transformer (MMDiT) with a sample-level router, which assigns each training sample to either action prediction or an auxiliary objective and thus makes supervision allocation explicit. Within this framework, we instantiate three LA targets based on LAPA and its DINO-feature variants, and three FV targets based on DINOv3, V-JEPA 2, and Wan2.2 representations. Across LIBERO, CALVIN, and four real-robot tasks, suitable LA and FV supervision outperforms action-only training under the same budget, even when action prediction receives no more than half of the training samples. FV generally outperforms LA, although the strongest LA target remains competitive on CALVIN. Diagnostic analyses suggest that DINO-based LA targets emphasize relative motion, whereas FV targets emphasize absolute position and task identity. We therefore argue that what robot policies learn to predict, and how training is allocated among these targets, should be treated as explicit design choices rather than defaulting to action-dominant training.
Tsinghua AI