Search papers, labs, and topics across Lattice.
This paper introduces GenPrior, a novel framework that leverages generative priors from pre-trained Text-to-Motion models to enhance zero-shot skeleton-based action recognition (ZSAR). By employing Dispersion-Gated Feature Fusion, GenPrior effectively integrates kinematic prototypes with textual embeddings, addressing the semantic-kinematic gap that hampers existing methods. Extensive evaluations on benchmark datasets reveal that GenPrior achieves state-of-the-art performance in both zero-shot and generalized zero-shot scenarios, significantly improving recognition accuracy for unseen action categories.
Bridging the semantic-kinematic gap, GenPrior achieves unprecedented accuracy in zero-shot action recognition by integrating generative motion priors with textual embeddings.
Zero-shot skeleton-based action recognition (ZSAR) aims to recognize unseen action categories by aligning skeleton features with textual semantics. However, existing methods rely on text-derived prototypes that inherently lack geometric structure and physical constraints, resulting in a pronounced \textit{semantic-kinematic gap}. To bridge this gap, we propose \textbf{GenPrior}, the first framework to exploit generative priors from pre-trained Text-to-Motion (T2M) models for ZSAR. Specifically, we introduce Dispersion-Gated Feature Fusion, which distills kinematic prototypes and intra-class dispersion from generative motion sequences and employs a learned gating network to adaptively inject reliable structural cues into textual embeddings while suppressing synthetic artifacts. Furthermore, we propose Generative Prototype Refinement, which leverages these generation-enhanced prototypes as anchors to mine high-confidence unseen samples, calibrating class prototypes toward the true distribution and thereby unleashing strong performance gains. Extensive experiments on NTU-60, NTU-120, and PKU-MMD demonstrate that GenPrior achieves state-of-the-art performance under both zero-shot and generalized zero-shot settings. Code is available at https://github.com/jidongkuang/GenPrior.
