Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces a top-down approach for multi-view multi-person human reconstruction that integrates camera calibration, cross-view association, and reconstruction into a unified framework. Utilizing cross-modal contrastive learning, the method encodes geometric structure, visual appearance, and semantic cues in a shared 3D space, allowing for improved correspondence reasoning and robustness against occlusions. Experimental results show significant advancements in accuracy and efficiency when reconstructing humans in complex real-world environments compared to traditional bottom-up methods.
Achieving robust multi-person reconstruction in challenging environments by leveraging a unified 3D representation that enhances cross-view consistency and occlusion handling.
Multi-view human reconstruction has been extensively studied under simplified settings, yet robust and efficient multi-person reconstruction in unconstrained environments remains challenging. Existing bottom-up methods often rely on accurate camera calibration and explicit cross-view matching, and therefore struggle with severe occlusions and ambiguities. We propose a new top-down paradigm that maintains a unified, instance-centric human-aware 3D space, enabling simultaneous camera calibration, cross-view association, and human reconstruction via cross-modal contrastive learning. Observations from multiple views are lifted and fused into this shared 3D space, where geometric structure, visual appearance, and human-centric semantic cues are jointly encoded at the instance level. We further introduce a spatial contrastive learning strategy that aligns 3D features corresponding to the same human instance across different views and modalities while separating different instances. This enables correspondence reasoning, semantic aggregation, and instance discrimination to be performed natively in 3D, improving cross-view consistency and robustness under severe occlusions. Finally, structured human body models are recovered in a feed-forward manner by regressing SMPL parameters from instance-level 3D human tokens. Extensive experiments demonstrate robust, accurate, and efficient multi-view human reconstruction in challenging real-world scenarios.