Search papers, labs, and topics across Lattice.
This paper introduces DynaMOMA, a framework that integrates instantaneous grasp trajectory prediction with whole-body control for mobile manipulation of dynamic objects. By employing an anchor-based diffusion model, the system generates temporally consistent grasp trajectories based on historical observations, which are then utilized by a reinforcement learning policy for dynamic grasping. Experimental results in both simulation and real-world settings demonstrate the method's robust performance and generalizability across various manipulation tasks.
Instantaneous grasp trajectory predictions enable robots to adeptly handle dynamic objects, achieving unprecedented performance in mobile manipulation tasks.
Mobile manipulation is a fundamental robotics task and has advanced rapidly in recent years, enabling robots to navigate, reach, and interact with objects in complex environments. However, mobile manipulation of dynamic objects remains highly challenging, as robots must coordinate the mobile base and arm while adapting to continuously evolving target poses. A key challenge lies in predicting temporally consistent short-horizon grasp trajectories from dynamic observations. In this work, we propose \ours{}, a dynamic mobile manipulation framework that couples instantaneous grasp trajectory prediction with whole-body control policy. Our predictor uses an anchor-based diffusion model to generate temporally consistent short-horizon grasp trajectories conditioned on historical observations. The predicted trajectories are then encoded as compact features and fed to a whole-body reinforcement learning policy, which controls the mobile manipulator for dynamic grasping. We further introduce a anticipation-guided reward that equips the policy with an anticipatory grasping horizon by adaptively shifting the target from the current grasp observation to the instantaneously predicted grasp trajectory. Through extensive experiments in Isaac Gym simulation, we show that our method achieves strong performance in mobile manipulation of dynamic objects across diverse settings and grasping metrics. Furthermore, our predictor and policy demonstrate strong generalizability in real-world experiments.
