Search papers, labs, and topics across Lattice.
This paper introduces the EgoHTR dataset, which consists of 55 scene-aligned 4D human motion sequences captured in complex environments using a multi-sensor setup. By addressing the limitations of existing datasets that fail to capture human-scene interactions, the authors provide a benchmark for human motion analysis and synthesis, achieving state-of-the-art accuracy when evaluated against motion-capture ground truth. Additionally, the dataset is utilized to train locomotion policies, successfully demonstrating hardware deployment on a Unitree G1 robot, paving the way for context-aware humanoid robots in unstructured terrains.
A groundbreaking dataset of 4D human motion sequences reveals how humanoid robots can effectively navigate complex terrains, setting a new standard for motion synthesis accuracy.
Deploying humanoid robots in unstructured terrain remains an open problem. While classic reinforcement learning struggles with the sheer complexity of real-world interactions, more promising methods leveraging human priors remain limited to models lacking contextual awareness. The restricted motion synthesis is a direct consequence of existing dataset pipelines failing to capture human-scene sequences in challenging environments. To bridge this gap between humanoid learning and scene reconstruction, we introduce the Egocentric Human-Terrain Reconstruction (EgoHTR) dataset. We develop and open-source a reconstruction pipeline capturing 55 scene-aligned 4D human motion sequences in diverse, complex environments using a multi-sensor setup of egocentric wearables and a portable 3D scanner. The resulting dataset comprises over 150k frames, which we evaluate against motion-capture ground truth, demonstrating state-of-the-art accuracy and establishing a rigorous benchmark for human motion analysis and synthesis. Further, we leverage this data to train perceptive locomotion policies, demonstrating hardware deployment on a Unitree G1 for reconstructed reference motions. Our pipeline enables community-driven dataset extensions and factors the problem to help researchers build foundational, context-aware robots that reliably traverse uneven terrain.
