Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces SimVLA, an end-to-end framework for training Visual-Language Agents (VLAs) for mobile manipulation entirely on synthetic data, addressing the prohibitive cost of real-world data collection. By pre-training on two novel simulation-derived datasets (SimAction and SimVQA) and post-training on a mixture including SimDeploy, SimVLA achieves zero-shot transfer to real-world mobile manipulation tasks. The framework outperforms policies trained on 50 in-domain real-world demonstrations, validating simulation's scalability for sim-to-real VLA learning.
Zero-shot sim-to-real mobile manipulation is now achievable with VLAs trained exclusively on synthetic data, outperforming real-world trained policies.
Large-scale, diverse datasets have driven the success of LLMs and VLMs. But VLAs for robotics remain limited by the cost and complexity of real-world data collection. While simulation offers a scalable alternative, its potential for sim-to-real VLA learning in mobile manipulation remains largely underexplored. We introduce SimVLA, an end-to-end framework that trains VLAs entirely on synthetic simulation data without teleoperation for mobile manipulation. SimVLA is first pre-trained on two complementary simulation-derived datasets: SimAction, a large-scale robot action dataset spanning 35 diverse mobile manipulation tasks, generated by composing atomic skills, and SimVQA, which leverages privileged simulator state to provide spatial, geometric, and subtask-level visual-language supervision. We further post-train SimVLA on a mixture of SimAction and SimDeploy, a dataset collected from policy rollouts across diverse simulated environments. We evaluate SimVLA on tasks including restocking, pouring, and cleaning, and show zero-shot transfer to real-world mobile manipulation, including real home environments. SimVLA outperforms policies trained on 50 in-domain real-world demonstrations, suggesting that simulation can enable scalable sim-to-real mobile manipulation. We further demonstrate the value of multiple complementary forms of supervision for effectively leveraging simulation in VLA training.