Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper addresses the challenges of EgoExo proficiency estimation by introducing a redundancy-aware approach that integrates egocentric and exocentric motion cues. The authors identify that simply increasing the number of exocentric views can degrade performance due to redundancy and overfitting, leading to ineffective learning. Their proposed modules, AdaMVS and VIB-GB, effectively select and fuse the most informative views, resulting in state-of-the-art performance on benchmark datasets EgoExo-4D and EgoExo-Fitness.
Adding more views can actually hurt performance—this research reveals how to intelligently fuse motion cues for superior action quality assessment.
EgoExo proficiency estimation aims to assess action quality by integrating fine-grained motion cues from egocentric (1st-person) views with spatial context from multiple exocentric (3rd-person) views. Simply adding more exocentric views degrades EgoExo performance, as redundant or noisy perspectives dilute useful motion cues. Our analysis identifies two key causes: (1) Multiview redundancy - From the data perspective, certain views provide limited or noisy information, diluting discriminative cues; (2) Overfitting - From the feature perspective, conventional fusion increases representational complexity, causing the model to memorise view-specific patterns rather than learn generalisable representations. To address these issues, we propose two complementary modules: AdaMVS, which adaptively identifies and fuses the most informative view tokens under weak supervision from the data perspective, and VIB-GB, which combines Gradient Blending and Variational Information Bottleneck regularisation from the feature perspective to compress redundant signals and suppress overfitting during training. Experiments on EgoExo-4D and EgoExo-Fitness demonstrate that our method learns both which view to look at and how to fuse them, achieving new state-of-the-art results. Our source code is available at https://github.com/dx199771/AdaMVS