Search papers, labs, and topics across Lattice.
This paper investigates using facial movement dynamics, captured via webcam and OpenPose, to monitor cognitive workload during a multitasking simulation. They extracted linear kinematics and recurrence quantification features from facial keypoints and trained random forest classifiers to predict workload levels. Results show that facial movement dynamics are sensitive to workload changes, achieving 85% accuracy in within-participant classification and 73% with minimal calibration, but generalizability across participants remains a challenge.
Facial micro-movements betray your cognitive load, revealing a new pathway to real-time workload monitoring using just a webcam.
Real-time cognitive workload monitoring is crucial in safety-critical environments, yet established measures are intrusive, expensive, or lack temporal resolution. We tested whether facial movement dynamics from a standard webcam could provide a low-cost alternative. Seventy-two participants completed a multitasking simulation (OpenMATB) under varied load while facial keypoints were tracked via OpenPose. Linear kinematics (velocity, acceleration, displacement) and recurrence quantification features were extracted. Increasing load altered dynamics across timescales: movement magnitudes rose, temporal organisation fragmented then reorganised into complex patterns, and eye-head coordination weakened. Random forest classifiers trained on pose kinematics outperformed task performance metrics (85% vs. 55% accuracy) but generalised poorly across participants (43% vs. 33% chance). Participant-specific models reached 50% accuracy with minimal calibration (2 minutes per condition), improving continuously to 73% without plateau. Facial movement dynamics sensitively track workload with brief calibration, enabling adaptive interfaces using commodity cameras, though individual differences limit cross-participant generalisation.
