Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces RhythmJEPA, a novel rhythm-structured joint-embedding predictive learning framework for remote photoplethysmography (rPPG), which addresses the limitations of existing self-supervised methods that focus on facial appearance rather than physiological dynamics. By employing a Cyclic Rhythm-State Planner (CRSP) and a Dual Order Mamba Encoder (DOM), RhythmJEPA effectively captures both local temporal continuity and long-range rhythm-consistent dependencies in pulse signals. Experimental results demonstrate that RhythmJEPA outperforms existing rPPG methods on benchmark datasets, highlighting its potential for more accurate physiological signal estimation from facial videos.
RhythmJEPA achieves superior rPPG performance by leveraging a rhythm-structured approach that captures the cyclic nature of pulse signals, outperforming traditional methods that overlook this critical aspect.
Remote photoplethysmography (rPPG) estimates physiological signals from facial videos by analyzing subtle pulse induced skin color variations. Despite recent progress, existing self-supervised rPPG methods mainly reconstruct masked pixels or low-level visual representations, which can bias the model toward facial appearance rather than latent physiological dy namics. Moreover, most recent Mamba-based approaches scan facial video tokens only in chronological order, limiting their ability to exploit the cyclic structure of pulse signals. To ad dress these limitations, we propose RhythmJEPA, a rhythm structured joint-embedding predictive learning framework for rPPG. Instead of reconstructing RGB frames, RhythmJEPA predicts latent teacher representations from masked facial videos, thereby encouraging physiology-aware representation learning in the embedding space. To explicitly model pulse-related tem poral structure, we introduce a Cyclic Rhythm-State Plan ner (CRSP), which estimates frame-wise latent physiological states and decodes the most plausible cyclic state path via dynamic programming with a constrained transition grammar. Guided by the decoded states, we further design a Dual Order Mamba Encoder (DOM), which combines conventional chronological scanning with state-ordered scanning to capture both local temporal continuity and long-range rhythm-consistent dependencies. Finally, a lightweight Spatial Pulse Mixer (SPM) extracts compact pulse-sensitive facial tokens with a favorable balance between complexity and performance. Experiments on PURE, UBFC-rPPG, and MMPD show competitive performance over representative rPPG methods. The codes are available at https://github.com/deconasser/RhythmJEPA.