Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces VAIC, a novel framework for humanoid robots that enables agile object interaction in unstructured environments by utilizing a decoupled command interface and relying on onboard depth and historical proprioception. The method addresses the limitations of existing controllers that depend on dense reference trajectories and perfect state observability, which hinder generalization in real-world applications. Experimental evaluations show that VAIC allows a single policy to effectively perform a variety of dynamic tasks, such as box carrying and skateboarding, outperforming traditional baselines and enhancing the deployment potential of humanoid robots.
VAIC enables humanoid robots to perform complex tasks in real-world settings without the need for perfect state observability, significantly advancing their practical deployment.
Humanoid robots hold immense potential for real-world assistance, yet agile interaction with objects in unstructured environments demands tightly coupled whole-body coordination. Despite recent advancements, current controllers face a critical deployment gap. They rely heavily on dense reference trajectories and perfect state observability, which inherently limits physical generalization. We present Vision Guided Agile Interaction Control (VAIC), a unified framework that bridges this gap by operating exclusively on onboard depth, historical proprioception, and a decoupled user command interface. VAIC employs a two-stage distillation paradigm. First, a privileged teacher policy masters diverse interaction skills using precise object kinematics and exact environmental states. Second, a deployable student policy distills these capabilities by replacing full body tracking with velocity targets across multiple axes and an interaction indicator for each frame. The student utilizes a recurrent object adaptation module to implicitly infer unobservable object dynamics from raw depth streams and proprioception. Evaluations and real-world deployments on the humanoid robot demonstrate that a single VAIC policy successfully executes highly diverse dynamic tasks. These tasks include box carrying, cart interaction, and skateboarding, consistently outperforming baselines and advancing autonomous humanoid deployment.
Tsinghua AI