Search papers, labs, and topics across Lattice.
This paper introduces the MVMP-HMR benchmark, a large-scale synthetic dataset designed for multiview multi-person human mesh recovery, featuring 15 complex scenes with up to 50 camera views and 30 interacting individuals. The authors propose a novel MVMP-HMR model that integrates multiview features into a 3D feature volume and utilizes a 3D pose estimation network to extract person-specific queries, addressing the challenges posed by severe occlusions. Experimental results reveal that existing state-of-the-art methods underperform on this benchmark, while the proposed model consistently achieves superior performance in recovering human meshes in complex environments.
Existing HMR methods falter under severe occlusions, but the MVMP-HMR model sets a new standard for multiview multi-person mesh recovery in large-scale scenes.
Human mesh recovery (HMR) aims to recover 3D human meshes from images. Most existing HMR benchmarks and methods focus on either multi-person reconstruction from a single view or single-person reconstruction from multiple views, where the number of subjects and the scene scale are relatively limited. Such settings are insufficient for real-world applications with large scenes and severe inter-person occlusions. To address this limitation, we introduce a large-scale synthetic benchmark for multiview multi-person HMR, termed MVMP-HMR. The proposed dataset contains 15 complex scenes with up to 50 camera views and 30 interacting persons, featuring large spatial coverage and severe occlusions, which significantly increases the difficulty of human mesh recovery. Based on this benchmark, we further propose a multiview multi-person whole-body human mesh recovery model, referred to as MVMP-HMR model. The model first fuses multiview features into a scene-level 3D feature volume, and then leverages pelvis joints predicted by a 3D pose estimation network to extract person-specific queries from the 3D feature volume. These human queries are cross-attended with the 3D feature volume and integrated to decode each person's 3D mesh. Moreover, we introduce two novel losses--the orientation loss and the 3D joint density loss--to alleviate orientation and pose ambiguities under severe occlusions. Experiments demonstrate that existing state-of-the-art HMR methods struggle on the proposed MVMP-HMR benchmark, while our method consistently outperforms prior SOTAs in large-scale scenes with severe occlusions.
