Track manipulation, locomotion and embodied systems.
Search papers, labs, and topics across Lattice.
To overcome the tendency of skeleton-based action models to over-index on dominant movements and deterministic coordinate regression, DiMoP couples masked joint diffusion modeling with self-supervised frame-level pseudo-classification. Conditioning the denoising of masked joints on visible context allows the model to uniformly capture subtle, moderate, and large kinematic dynamics across noise scales while maintaining sequence-level temporal coherence. The resulting joint generative-discriminative representations achieve state-of-the-art performance on NTU RGB+D 60/120 and PKUMMD, improving NTU RGB+D 120 cross-subject accuracy by 1.1 percentage points over prior baselines.
Replacing deterministic coordinate regression with masked joint diffusion captures the subtle, long-tail dynamics standard skeleton encoders miss, driving a 1.1% gain on competitive action recognition benchmarks.
Robust skeleton-based action recognition requires representations that capture a wide spectrum of motions, from subtle to moderate and strong ones. Existing methods often focus on strong motions. This paper introduces DiMoP, a masking- and diffusion-driven motion representation learning method with frame-level pseudo-classification to explicitly learn the distribution of joint motions rather than regressing deterministic coordinates, as existing methods often do. By diffusing masked joints with progressive noise and denoising them conditioned on visible joints, DiMoP learns through controllable noising and denoising processes, enabling uniform learning of weak, moderate, and strong dynamics. To enable the masking-based generative diffusion learning with a discriminative capability, a pseudo-frame classifier is proposed that enforces the learning towards sequence-consistent and temporally coherent pseudo-labels without manual annotations. Together, these strategies provide a principled mechanism for joint generative and discriminative motion modeling. DiMoP achieves state-of-the-art performance across NTU RGB+D 60/120, and PKUMMD, including a 1.1 percentage point gain over prior works on NTU RGB+D 120 with the cross-subject protocol.