Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces Multi-view Expected-OKS Maximization (MEOM), a novel approach for 3D human pose estimation that leverages the entire spatial distribution of 2D keypoint heatmaps rather than collapsing them into single peaks. By fusing heatmaps across views and assessing their reliability through highest-density-region calibration, the method significantly improves accuracy in pose triangulation, especially in challenging scenarios like occlusion. The results show that MEOM achieves competitive performance with state-of-the-art methods using fewer resources, particularly excelling in both 3D-supervised and unsupervised settings.
Achieving 19.11 mm absolute MPJPE on Human3.6M, MEOM outperforms volumetric approaches at half the inference cost, redefining efficiency in 3D human pose estimation.
Conventional algebraic triangulation solves 3D human pose estimation (HPE) from multi-view 2D keypoints. The typical approach, decoding 2D keypoints from predicted heatmaps, is unreliable as heatmaps can be multimodal under occlusion, and collapsing them into single peaks discards their spatial distribution. We seek to use the entire heatmap to estimate 3D poses more accurately, which requires solving two problems: how to robustly fuse heatmaps across views, and how to assess the reliability of heatmaps. For the former, we introduce a novel objective, Multi-viewExpected-OKS Maximization (MEOM), that locates a 3D joint where the views agree in probability mass. For the latter, we adopt highest-density-region (HDR) calibration as a diagnostic of that mass, independently of distance-based metrics. The proposed framework covers two settings, with and without 3D supervision. Without 3D supervision, we optimize 3D poses from pretrained heatmap predictors by maximizing MEOM, achieving comparable performance with state-of-the-art methods that rely on larger backbones, temporal fusion, and simulated 3D data. On ambiguous Human3.6M (H36MA) and occluded CMU Panoptic frames, the advantage is substantial. When 3D labels are available, we train the model end-to-end with a combined MEOM and MSE loss, achieving 19.11 mm absolute MPJPE on Human3.6M outperforming the state-of-the-art volumetric approach on absolute MPJPE at half the inference cost.