Search papers, labs, and topics across Lattice.
The Ambient Capture Engine (ACE) addresses the critical data bottleneck in embodied intelligence by creating a comprehensive human-centric data engine that captures first-person perception and whole-body motion in real home environments. By utilizing both table-scale and room-scale configurations, ACE records a unified multisensory stream, resulting in the ACE-Data-0 dataset, which includes 150 hours of video across 200 task categories performed by 50 participants. Evaluations reveal significant performance gaps in existing methods, highlighting the potential of ACE-Data-0 to enhance imitation learning and embodied AI applications through its rich, synchronized data.
ACE-Data-0 reveals that existing models struggle significantly with complex interactions, exposing critical gaps in embodied AI performance.
Embodied intelligence faces a fundamental data bottleneck. Models must capture how first-person perception, whole-body motion, dexterous manipulation, object state, sound, and touch evolve together as humans pursue goals over time. Existing datasets fragment this experience across viewpoints, modalities, or spatial scales, leaving the full perception-action loop only partially observed. We introduce the Ambient Capture Engine (ACE), a human-centric data engine that transforms real home environments into spatially calibrated, temporally synchronized recording studios. ACE operates at two complementary scales: a table-scale configuration resolves hand-object manipulation, while a room-scale configuration captures whole-body motion, locomotion, and interactions across a furnished home. ACE records egocentric and multi-view exocentric video, full-body and articulated hand motion, object geometry and 6-DoF trajectories, audio, and tactile signals as a unified multisensory stream. Using ACE, we build ACE-Data-0, comprising 150 hours and 17M video frames across 200 task categories, performed by 50 participants in 2 environments, for a total of 75,000 interaction episodes. The dataset spans atomic manipulation, long-horizon chains of household activities, and human-scene interaction, while preserving natural behavioral variation through goal-level rather than step-by-step instructions. We further introduce a hierarchical benchmark that progresses from signals to scene components and then to interactions. Evaluations of state-of-the-art methods expose substantial gaps under contact, occlusion, egomotion, and long temporal horizons. ACE-Data-0 provides synchronized human demonstrations with aligned perceptual, kinematic, and contact supervision, offering a scalable foundation for imitation learning, world models, vision-language-action systems, and embodied AI.
