Search papers, labs, and topics across Lattice.
The paper introduces HAIC, a framework for humanoid robots to interact with underactuated objects having independent dynamics, addressing limitations of prior HOI methods focused on rigidly coupled objects. HAIC uses a dynamics predictor to estimate high-order object states from proprioceptive history, projecting these onto geometric priors to create a dynamic occupancy map for collision avoidance and contact affordance inference. Through asymmetric fine-tuning of a world model, HAIC achieves robust performance on agile manipulation tasks like skateboarding and cart pushing, as well as long-horizon multi-object tasks.
Humanoid robots can now perform agile tasks like skateboarding and cart-pushing with underactuated objects, thanks to a dynamics-aware world model that predicts object states from robot's own sensory history.
Humanoid robots show promise for complex whole-body tasks in unstructured environments. Although Human-Object Interaction (HOI) has advanced, most methods focus on fully actuated objects rigidly coupled to the robot, ignoring underactuated objects with independent dynamics and non-holonomic constraints. These introduce control challenges from coupling forces and occlusions. We present HAIC, a unified framework for robust interaction across diverse object dynamics without external state estimation. Our key contribution is a dynamics predictor that estimates high-order object states (velocity, acceleration) solely from proprioceptive history. These predictions are projected onto static geometric priors to form a spatially grounded dynamic occupancy map, enabling the policy to infer collision boundaries and contact affordances in blind spots. We use asymmetric fine-tuning, where a world model continuously adapts to the student policy's exploration, ensuring robust state estimation under distribution shifts. Experiments on a humanoid robot show HAIC achieves high success rates in agile tasks (skateboarding, cart pushing/pulling under various loads) by proactively compensating for inertial perturbations, and also masters multi-object long-horizon tasks like carrying a box across varied terrain by predicting the dynamics of multiple objects.
