Search papers, labs, and topics across Lattice.
The paper introduces InCoM, a framework for whole-body mobile manipulation that addresses challenges in coordinated control and perceptual attention allocation. InCoM infers latent motion intent to reweight multi-scale perceptual features and uses a geometric-semantic structured alignment mechanism for robust cross-modal perception. A decoupled coordinated flow matching action decoder is designed to explicitly model coordinated base-arm action generation, simplifying control optimization.
Achieve up to 28% better success rates in whole-body mobile manipulation by decoupling base and arm control while intelligently allocating perceptual attention.
Whole-body mobile manipulation is a fundamental capability for general-purpose robotic agents, requiring both coordinated control of the mobile base and manipulator and robust perception under dynamically changing viewpoints. However, existing approaches face two key challenges: strong coupling between base and arm actions complicates whole-body control optimization, and perceptual attention is often poorly allocated as viewpoints shift during mobile manipulation. We propose InCoM, an intent-driven perception and structured coordination framework for whole-body mobile manipulation. InCoM infers latent motion intent to dynamically reweight multi-scale perceptual features, enabling stage-adaptive allocation of perceptual attention. To support robust cross-modal perception, InCoM further incorporates a geometric-semantic structured alignment mechanism that enhances multimodal correspondence. On the control side, we design a decoupled coordinated flow matching action decoder that explicitly models coordinated base-arm action generation, alleviating optimization difficulties caused by control coupling. Without access to privileged perceptual information, InCoM outperforms state-of-the-art methods on three ManiSkill-HAB scenarios by 28.2%, 26.1%, and 23.6% in success rate, demonstrating strong effectiveness for whole-body mobile manipulation.
