Track manipulation, locomotion and embodied systems.
Search papers, labs, and topics across Lattice.
The authors investigate the feasibility of full-body 3D pose estimation from a single consumer earbud IMU by adapting recurrent kinematic architectures (IMUPoser and MobilePoser) evaluated against synchronized multi-view RGB-D pseudo-ground-truth. A single head-mounted sensor achieves 79.0 mm rigid-MPJPE lower-body pose and 0.809 macro-F1 foot-contact accuracy, maintaining performance even in a causal streaming configuration. Crucially, the benchmark reveals that adding consumer foot insoles fails to improve and often degrades pose accuracy due to orientation drift, establishing that sensor noise—not sensor quantity—is the primary bottleneck in sparse wearable tracking.
Counterintuitively, a single off-the-shelf earbud IMU tracks 3D lower-body kinematics at 79 mm error, whereas adding commercial insole sensors actually degrades pose reconstruction due to orientation noise.
Consumer earbuds already stream inertial motion data from the head, one of the most widely worn sensor locations on the body. We ask how much of the 3D body pose a single such head IMU can recover, and whether adding more consumer sensors actually helps. We build a multimodal capture pipeline that records four-view RGB-D video together with an AirPods head IMU and two Striv insole IMUs, synchronize the streams post-hoc, and generate pseudo-ground-truth with SAM 3D Body, yielding a 35-take single-subject benchmark spanning gait, turning, vertical, everyday, and clinically inspired motions. Adapting two recurrent model families (IMUPoser and MobilePoser), we show that one head IMU recovers lower-body pose at 79.0 mm rigid-MPJPE and per-foot ground contact at 0.809 macro-F1, and that a causal variant retains most of this accuracy at streaming latency. In paired per-take significance tests across both families, adding the consumer foot IMUs never significantly improves pose and significantly degrades it in two of four model-split combinations; a mounting-bias probe and feet-only ablation identify insole orientation quality, not foot placement, as the mechanism. Extending the output to a 20-joint full-body skeleton maps the boundary: gross distal-arm motion is partially recoverable from the head alone, proximal upper-body pose is not, and staged fine-tuning recovers the leg accuracy that naive joint training sacrifices to multi-task dilution. For learned pose from consumer wearables, sensor reliability, not sensor count, is the binding constraint here. For the devices tested, the earbud is its sweet spot. Code is available at https://github.com/ZhilinGuo/one-sensor-whole-body.