Search papers, labs, and topics across Lattice.
The paper introduces FootMR, a method to refine foot motion in markerless monocular human motion capture by lifting 2D foot keypoint sequences to 3D, addressing the limitations of existing methods in capturing fine-grained foot articulations. FootMR leverages large-scale motion capture data by avoiding direct image input and incorporates knee and foot motion context to resolve 2D-to-3D lifting ambiguities, predicting residual foot motion. Experiments on the new MOOF dataset, as well as MOYO and RICH, demonstrate that FootMR significantly outperforms state-of-the-art methods, reducing ankle joint angle error on MOYO by up to 30% compared to video-based approaches.
Fine-grained foot motion capture, a notoriously hard problem, gets a 30% accuracy boost by cleverly lifting 2D keypoints to 3D using motion capture data and contextual information, bypassing the need for direct image-3D annotation pairs.
State-of-the-art methods can recover accurate overall 3D human body motion from in-the-wild videos. However, they often fail to capture fine-grained articulations, especially in the feet, which are critical for applications such as gait analysis and animation. This limitation results from training datasets with inaccurate foot annotations and limited foot motion diversity. We address this gap with FootMR, a Foot Motion Refinement method that refines foot motion estimated by an existing human recovery model through lifting 2D foot keypoint sequences to 3D. By avoiding direct image input, FootMR circumvents inaccurate image-3D annotation pairs and can instead leverage large-scale motion capture data. To resolve ambiguities of 2D-to-3D lifting, FootMR incorporates knee and foot motion as context and predicts only residual foot motion. Generalization to extreme foot poses is further improved by representing joints in global rather than parent-relative rotations and applying extensive data augmentation. To support evaluation of foot motion reconstruction, we introduce MOOF, a 2D dataset of complex foot movements. Experiments on MOOF, MOYO, and RICH show that FootMR outperforms state-of-the-art methods, reducing ankle joint angle error on MOYO by up to 30% over the best video-based approach.
