Search papers, labs, and topics across Lattice.
This paper introduces EgoHRV, a novel method for estimating heart rate variability (HRV) and heart rate (HR) from egocentric gaze video, addressing the challenge of capturing fine-grained physiological indicators amidst motion and noise. By employing a 3D backbone and a low-high decomposition module, EgoHRV effectively extracts the blood volume pulse (BVP) signal, leveraging cross-domain pretraining to align camera-derived signals with traditional contact-based measurements. The method achieves state-of-the-art performance in HR and HRV estimation, enhancing behavioral modeling accuracy by 17.8% when integrated into existing systems, thereby facilitating stress and arousal-aware applications in egocentric platforms.
EgoHRV reveals that gaze video can be a powerful tool for continuous heart rate variability estimation, unlocking new dimensions in behavioral analysis.
Egocentric vision systems capture human behavior from visible cues, but overlook physiological indicators of autonomic states such as stress, engagement, and attention. Heart rate variability (HRV) is a widely used noninvasive marker of autonomic regulation under stress. HRV reflects small timing differences between successive heartbeats and has so far been out of reach for egocentric platforms, where motion and noise in gaze video mask exactly this fine-grained timing. We propose EgoHRV, a method that estimates HRV as well as heart rate (HR) from the gaze cameras that are already integrated into egocentric headsets. Our pipeline combines a 3D backbone with a novel low--high decomposition module that extracts the blood volume pulse (BVP) signal from gaze video. Our cross-domain pretraining aligns the frequency-domain representations of contact-based and camera-derived signals. This alignment gives EgoHRV the temporal precision to recover HRV from the subtle fluctuations in gaze video. EgoHRV achieves state-of-the-art accuracy for HR and HRV estimation from egocentric video, and its uncertainty-aware design improves downstream behavioral modeling. Integrating our HRV estimates and confidence measures into EgoExo4D's proficiency estimator raises accuracy by 17.8%. Beyond skill, continuous HRV estimation also opens egocentric systems to stress- and arousal-aware estimation tasks. Code: https://github.com/eth-siplab/EgoHRV
