Track manipulation, locomotion and embodied systems.
Search papers, labs, and topics across Lattice.
To resolve ego-motion perception drift and heterogeneous action coordination in mobile manipulation, the authors built MM-ABC, a foundation model pretrained on over 5,000 hours of multi-embodiment robot data. The framework combines sparse multi-level VLM features with MM-APT, a dual-stream architecture that coordinates decoupled arm and base actions using masked joint attention, clean-action prediction, and training-only future geometric imagination. The resulting model achieves 44.71% success on EBench, 61.2% on RoboCasa365, and an 83% mean success rate across five physical real-world manipulation tasks.
Mobile manipulation policies do not need joint action spaces to coordinate effectively: decoupling arm and base streams via masked joint attention and training-time world imagination drives state-of-the-art coordination across 17 distinct embodiments.
Mobile manipulation extends robot interaction beyond a fixed kinematic workspace by making the reachable region itself controllable. This flexibility introduces two central challenges: spatially grounded perception under continuous ego-motion and coordinated control of heterogeneous arm and base actions. Existing approaches strengthen geometry through explicit 3D representations or predictive world models, and often decouple mobility and manipulation into separate action streams. We argue that effective mobile manipulation requires not only decoupling, but also representations that support efficient cross-stream collaboration. We present MM-ABC, a foundation model built around Seeing, Coordinating, and Imagining Arm-Base Collaboration. MM-ABC combines sparse multi-level VLM features for spatial perception; a training-only future branch that uses world imagination and geometric intent as extra supervision, strengthening perception and manipulation-intent prediction and improving the overall learning signal; and MM-APT, which coordinates separate manipulation and mobility streams through masked joint attention and clean-action x-prediction. In controlled ablations, replacing clean-action prediction with velocity prediction lowers success on RoboCasa365 composite-seen tasks from 32.8% to 29.2%, and removing future supervision or multilevel conditioning causes larger drops. We pretrain MM-ABC on 5,000+ hours of heterogeneous robot data spanning 400K+ episodes, 12 datasets, and 17 embodiments. Experiments cover EBench, RoboCasa365, ManiSkill-HAB, LIBERO, LIBERO-Plus, and real-world mobile manipulation. MM-ABC achieves 44.71% success on EBench, 61.2% on RoboCasa365, 99.1% on LIBERO, 82.8% on LIBERO-Plus without perturbation training, and 83% mean success on five real-world tasks.
Tsinghua AI