Search papers, labs, and topics across Lattice.
This paper introduces LingBot-VA 2.0, a video-action foundation model specifically designed for robot control, addressing the inadequacies of existing video generative models in physical environments. By implementing a semantic visual-action tokenizer, a causal pretraining paradigm, a sparse MoE backbone, and an enhanced asynchronous inference scheme, the model significantly improves instruction following and action precision. Real-world tests demonstrate its capability for few-shot generalization across complex manipulation tasks, establishing it as a robust solution for generalizable robot control.
LingBot-VA 2.0 achieves few-shot generalization in complex robot manipulation tasks, outperforming traditional video generative models.
The advent of video-action models offers a promising path for robot control. Nevertheless, we argue that repurposing video generative models designed for digital content creation is inherently inadequate for physical environments. To bridge this gap, we present LingBot-VA 2.0, a video-action foundation model built from the ground up for embodiment. Four core design principles showcase its evolution from LingBot-VA. (1) Departing from traditional reconstruction-focused VAEs, we introduce a semantic visual-action tokenizer, which aligns visual representations with both semantics and actions, improving instruction following and action precision in subsequent policy learning. (2) Given the strictly causal nature of temporal dynamics, we adopt a causal pretraining paradigm, training from scratch to circumvent the catastrophic forgetting that frequently occurs when adapting bidirectional architectures. (3) To meet the demands of high-frequency inference, our model employs a sparse MoE backbone, expanding model capacity without compromising efficiency. (4) Real-time closed-loop control is realized through an enhanced asynchronous inference scheme, which predicts future latents in parallel with action execution while re-grounding each rollout on the latest observation via learned forward dynamics. Real-world deployment validates LingBot-VA 2.0 as a robust foundation model, as evidenced by its few-shot generalization across complex manipulation tasks.
