Track trials, findings, and clinical evidence as it lands.
Search papers, labs, and topics across Lattice.
Dense4D tackles the challenge of dense 4D human correspondence by introducing a novel feed-forward model that predicts Human Point Maps (HPMs) and depth from monocular RGB images, eliminating the need for a body template. This approach enables amodal correspondence, localizing hidden surface points by leveraging observations across multiple frames, and establishes correspondence by matching HPM coordinates across person-centered crops. Experimental results show superior tracking accuracy and agreement with scanned surfaces compared to existing methods, demonstrating coherent human geometry reconstruction through articulation and clothing deformation.
A single feed-forward network reconstructs dense 4D human correspondence, including clothing and hair, from monocular RGB images without a body template, achieving amodal tracking and precise surface point localization.
Reconstructing human motion in 4D requires identifying the same surface points through articulation, deformation and changes in visibility. We introduce Dense4D, to our knowledge the first feed-forward model dedicated to dense 4D correspondence across the body, clothing and hair from a set of monocular RGB images, without a body template. Trained specifically for human geometry, Dense4D jointly processes person-centred crops with known image-to-crop transforms to predict Human Point Maps (HPMs) and depth for each image. An HPM assigns each person pixel the 3D position of its observed surface point at a common reference time. These maps support amodal correspondence: points hidden in the reference image can be localised from observations in other frames. Matching HPM coordinates across crops establishes correspondence without predicting camera poses or large scene displacements; per-image depth then lifts the matches into 3D tracks. Experiments on real people demonstrate substantially higher tracking accuracy and closer agreement with scanned surfaces than the evaluated baselines. Points from different frames assemble into coherent human geometry in the reference pose, providing visible evidence of consistent dense correspondence through articulation and clothing deformation. Additional qualitative results, videos, and an interactive viewer are available at https://alpguler.com/dense4d.