Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This study tackles the challenge of recognizing ambivalence and hesitancy (A/H) in video interviews by integrating affect-specialized text, audio, and visual features with a novel approach called Affective Marker Fusion (AMF). The researchers found that while language serves as the most significant predictor of A/H, the inclusion of ASR-erased time features—derived from speech recognition gaps—enhances the model's performance, achieving an average precision of 0.731 on the test set. Additionally, the findings highlight the importance of calibration over architectural complexity, revealing that a fixed decision threshold is more effective than overfitting ensemble weights on validation data.
Language is the dominant channel for detecting ambivalence and hesitancy, but leveraging ASR-erased time features can significantly boost recognition accuracy.
We address ambivalence and hesitancy (A/H) recognition in the ABAW 2026 BAH Challenge: given a short interview video, predict whether the person shows signs of A/H. Our system combines affect-specialised text, audio, and visual representations with a small set of readable linguistic hesitation cues, fused by a reliability gate we call Affective Marker Fusion (AMF), and finished with a simple AP-weighted ensemble at a fixed decision threshold. We also introduce \emph{ASR-erased time}: speech recognisers delete fillers and hesitation pauses from the transcript, but the chunk timestamps keep the time those events took, and sixteen features built from these gaps form the strongest and most independent non-verbal channel we measured (AP $0.718$, correlation $0.11$--$0.36$ with all other members). Across controlled experiments we find three things: cross-modal conflict design does not reliably help on BAH; language is by far the strongest channel while affect-specialised audio is a useful second; and calibration matters more than architecture. Fitting ensemble weights and a threshold on the small validation split overfits: it scores $0.741$ macro-F1 on validation but only $0.690$ on the untouched test set. AP-weighting at a fixed threshold instead reaches $\mathbf{0.731}$ on test.