Search papers, labs, and topics across Lattice.
This paper introduces the Mixture of Action Knowledge Experts (MoAKE) framework for Action Quality Assessment (AQA), which addresses the limitations of traditional models that evaluate actions one-by-one. By leveraging a unified model that learns from diverse action patterns and dynamically aggregates knowledge from complementary experts, MoAKE effectively mitigates negative knowledge transfer and enhances generalization across various action types. Experimental results on multiple datasets reveal that MoAKE not only outperforms existing AQA methods in an all-in-one context but also demonstrates robust performance in zero/few-shot scenarios.
MoAKE achieves superior action quality assessment by unifying diverse action evaluations into a single model, overcoming the limitations of traditional one-by-one approaches.
Action Quality Assessment (AQA) aims to objectively evaluate performance quality from action videos. Most existing methods follow a ``one-by-one''paradigm, training a separate model for each action type. This setting limits real-world deployment, as it requires prior action-type knowledge to select the corresponding model and suffers from poor generalization across diverse actions. To address these limitations, we study the challenging task of all-in-one AQA, which aims to assess heterogeneous actions within a single unified model. We propose a novel Mixture of Action Knowledge Experts (MoAKE) framework, designed to mitigate negative knowledge transfer caused by large semantic discrepancies among actions. MoAKE learns complementary experts that capture diverse action patterns within a shared semantic space and dynamically aggregates their knowledge to adapt the assessment to the input action. Each expert is tailored with segment-aware prototypes to handle varying temporal lengths, together with an Adaptive Intra- and Inter-Segment Relationship Modeling (AIISRM) module to model multi-granularity temporal dynamics. Furthermore, we establish comprehensive benchmarks for all-in-one as well as zero/few-shot AQA. Extensive experiments on three long-term datasets demonstrate that MoAKE significantly outperforms existing methods in the all-in-one setting, while also achieving consistent generalization on three short-term datasets under zero/few-shot evaluation. Code is available at https://github.com/XuHuangbiao/MoAKE.
