Search papers, labs, and topics across Lattice.
The CalTennis dataset provides a comprehensive benchmark for monocular-to-3D pose estimation, featuring over 11 million frames of synchronized multi-view recordings from tennis practice and matches. This dataset is significantly larger than existing datasets, allowing for more robust evaluation of pose estimation algorithms in real-world scenarios. Benchmarking reveals that while 3D joint angle recovery has improved, challenges remain in accurately estimating depth and foot contact, highlighting critical areas for further research and development.
Despite advancements in 3D joint angle recovery, existing models falter in depth estimation and foot contact, revealing crucial gaps in pose estimation performance.
The Caltech Tennis Dataset (CalTennis) is a large-scale video benchmark for evaluating monocular-to-3D pose estimation in the wild. CalTennis comprises over 11 million frames (51 hours) of tennis practice and match play from 40 players, captured with 2-6 synchronized cameras at 60 Hz. It is 10 times larger than existing in-the-wild human motion video datasets and 3 times larger than existing MOCAP-ground-truthed datasets, and it is the first large-scale benchmark to provide synchronized multi-view recordings of expert athletic motion. The multi-view setup enables inexpensive, label-free evaluation of monocular-to-3D pose estimation algorithms. We describe a simple, standardized protocol that enables data collection without specialized equipment or expertise, along with fully automated video calibration and synchronization. Benchmarking state-of-the-art monocular-to-3D pose methods on CalTennis, we find that while 3D joint angle recovery is now quite accurate, all models struggle to estimate depth and foot contact consistently. We further propose two novel performance metrics, footwork and stability, as well as qualitatively study body shape inconsistency. These metrics expose previously underexplored failure modes and point to concrete opportunities for improvement in pose estimation and action analysis.
