Search papers, labs, and topics across Lattice.
This paper introduces an end-to-end spatial-temporal transformer framework for remote heart-rate estimation using RGB cameras, addressing the significant challenge of illumination variation in real-world environments. By integrating advanced techniques such as PRNet-based 3D face alignment and a novel Residual Temporal Standardization Module, the proposed method achieves a remarkable heart-rate mean absolute error (MAE) of 0.79 bpm and a correlation of 0.982. The results demonstrate a 93.6% reduction in HR MAE compared to the PhysFormer baseline, showcasing the framework's robustness under varying lighting conditions.
Achieving a heart-rate estimation accuracy that withstands significant illumination changes could revolutionize physiological sensing in robots.
Physiological awareness is important for service, social, and assistive robots that interact with humans in everyday environments. Remote photoplethysmography (rPPG) enables non-contact heart-rate (HR) estimation from an RGB camera, making it a promising sensing modality for robot-mounted vision systems. However, illumination variation remains a major barrier to robust deployment. This paper presents an end-to-end spatial-temporal transformer framework for remote HR estimation on a new dataset with varied illumination. Our estimator integrates PRNet-based 3D face alignment, clip-level illumination augmentation, the Residual Temporal Standardization Module, and controlled hybrid temporal-frequency supervision. The training objective combines a Soft-Shifted Pearson waveform loss with a spectral Kullback-Leibler divergence loss, where a tuned weight ($\mathbf尾$) controls the contribution of frequency-domain heart-rate guidance. Experiments on a static all-level mix protocol covering three illumination levels show that $\mathbf尾=5$ provides the strongest result among the tested beta settings, achieving a best-run HR mean absolute error (MAE) of 0.79 bpm and an HR correlation of 0.982. Compared with the PhysFormer baseline evaluated on our dataset, our estimator reduces HR MAE by 93.6 %, while increasing HR correlation from 0.088 to 0.982, making it usable when illumination varies.
