Search papers, labs, and topics across Lattice.
This paper introduces DreamTrajectory, a novel trajectory-guided framework for language-conditioned mobile manipulation that addresses the limitations of existing Vision-Language-Action (VLA) policies. By jointly predicting an intention-level end-effector trajectory alongside a whole-body action chunk, DreamTrajectory enhances the precision of base-arm coordination, while a lightweight trajectory world model refines action selection by aligning candidate actions with the intended trajectory. The framework significantly improves success rates on both simulated and real-world tasks, achieving up to 90.0% success on complex manipulation scenarios.
Trajectory guidance boosts mobile manipulation success rates by over 20%, transforming how robots interpret and execute complex tasks.
Mobile manipulation requires a robot to coordinate base and arm motion under continuously changing viewpoints and contact conditions, within an action space far larger than that of fixed-base manipulation. Existing Vision-Language-Action (VLA) policies are limited in two respects. (i)They map observations directly to whole-body action chunks, searching this large action space without an explicit task-space motion plan, which makes coordinated base--arm prediction imprecise. (ii)They execute the predicted chunk open-loop, without checking whether the actions can realize the motion the policy intended, so control errors and unmodeled contacts accumulate into a gap between planned and realized motion. We present DreamTrajectory, a trajectory-guided framework for language-conditioned mobile manipulation that introduces one component for each limitation. Addressing(i), DreamTrajectory jointly predicts an intention-level end-effector trajectory and a whole-body action chunk in a single action expert, so that the trajectory explicitly guides base--arm action generation instead of remaining implicit. Addressing(ii), a lightweight trajectory world model predicts the trajectory that a candidate action chunk would induce, and a test-time search--predict--score procedure selects the candidate best aligned with the planned trajectory. On MS-HAB, trajectory guidance raises average success from 32.3% to 47.5% and test-time refinement further to 54.8%, with the largest gains on contact-rich articulated-object tasks. On three real-world mobile manipulation tasks, the corresponding average success rates are 63.3%, 81.7%, and 90.0%.
