Search papers, labs, and topics across Lattice.
This paper addresses the challenges of mobile manipulation in robotics by developing a whole-body teleoperation system and a novel panorama-aware vision-language-action (VLA) policy, named PanoVLA. The system facilitates the coordinated control of a wheeled bimanual robot through a VR interface and generates a comprehensive dataset of 5.5 hours of multimodal demonstrations. PanoVLA, which integrates panoramic observations with language instructions and robot states, significantly enhances performance in mobile manipulation tasks, achieving a 91.3% stage completion rate and a 73.4% end-to-end success rate, outperforming traditional local-view approaches.
Incorporating panoramic spatial context into VLA policies boosts mobile manipulation success rates by over 20% compared to local-view methods.
Mobile manipulation is a key capability for embodied intelligence, enabling robots to accomplish complex multi-stage tasks in open-world environments. However, mobile manipulation poses two key challenges for vision-language-action (VLA) policies: At the data level, the efficient collection of high-quality whole-body demonstrations demands the coordinated control of both the mobile base and the robotic arms; at the model level, existing VLA models predominantly rely on local camera observations, whose limited field of view hinders global spatial understanding. To address these challenges, we develop a whole-body teleoperation system and a panoramic-aware VLA policy. The system enables coordinated control of a wheeled bimanual robot through a single VR interface and supports the acquisition of a real-world mobile manipulation dataset comprising 5.5 hours of multimodal demonstrations. Building upon this dataset, we propose PanoVLA, a panorama-aware vision-language-action policy for mobile bimanual manipulation. Built upon a Mixture-of-Transformers architecture, PanoVLA introduces global spatial context through dedicated panorama encoding and fusion modules, enabling effective integration of panoramic observations with language instructions and robot states for action generation. Evaluation on four real-world mobile manipulation tasks demonstrates that PanoVLA achieves an average stage completion rate of 91.3\% and an end-to-end success rate of 73.4\%, substantially outperforming local-view baselines. These results demonstrate that incorporating panoramic spatial context improves spatial understanding and closed-loop manipulation performance in mobile robots.
