Search papers, labs, and topics across Lattice.
AnyBody introduces a unified whole-body humanoid controller that operates from arbitrary subsets of body keypoints, addressing limitations of traditional physics-based trackers that require full-body motion capture and hierarchical control. By training a privileged teacher tracker on a vast unstructured motion dataset, the authors distill a deterministic encoder-decoder model that utilizes a unit sphere latent space. The system's effectiveness is demonstrated through various applications, including free-form control and complex locomotion tasks, showcasing its ability to adapt to diverse keypoint inputs while maintaining coordinated motion.
AnyBody enables humanoid robots to seamlessly control movements from any chosen subset of body keypoints, revolutionizing how we approach humanoid locomotion and manipulation.
We present AnyBody, a unified whole-body humanoid controller driven by an arbitrary subset of body keypoints chosen at deploy time. Prior physics-based trackers either rely on expensive full-body motion capture and error-prone trajectory retargeting, which bottleneck scalable data collection and policy learning, or decompose upper- and lower-body control into separate hierarchical representations, sacrificing the coordinated whole-body motions that loco-manipulation requires. We close this gap by learning a single latent motion representation that any keypoint subset can address. To achieve this, we first train a privileged teacher tracker on a large unstructured motion corpus and distill it online into a deterministic encoder-decoder student whose latent space is a unit sphere. We then train a transformer keypoint encoder that admits any subset of body keypoints through masked self-attention, aligning it to the privileged latent. Additionally, we treat the frozen decoder as a motor prior and specialize downstream tasks with a lightweight residual corrector in the latent space. We demonstrate the effectiveness of AnyBody by tracking large-scale human motions from arbitrary keypoint subsets, free-form control, flexibly teleoperating, and learning downstream behaviors including locomotion, in-air writing, and obstacle-reach.
