Search papers, labs, and topics across Lattice.
This paper introduces the Multi-Level Speaker-Adaptive Network (ML-SAN) to enhance emotion recognition in conversations by addressing the variability in individual expressive traits. By employing a three-stage adaptive process鈥擨nput-level Calibration, Interaction-level Gating, and Output-level Regularization鈥擬L-SAN effectively mitigates speaker identity confusion and improves the model's ability to recognize emotions across diverse communicative styles. Experimental results on the MELD and IEMOCAP datasets demonstrate that ML-SAN outperforms existing models, particularly in recognizing nuanced emotional expressions in multi-turn dialogues.
Emotion recognition can be significantly improved by adapting to individual expressive traits, with ML-SAN outperforming static models in capturing nuanced emotional expressions.
To establish empathy with machines, it is essential to fully understand human emotional changes. However, research in multimodal emotion recognition often overlooks one problem: individual expressive traits vary significantly, which means that different people may express emotions differently. In our daily lives, we can see this. When communicating with different people, some express"happiness"through their facial expressions and words, while others may hide their happiness or express it through their actions. Both are expressions of'happiness,'but such differences in emotional expression are still too difficult for machines to distinguish. Current emotion recognition remains at a'static'level, using a single recognition model to identify all emotional styles. This"simplification"often affects the recognition results, especially in multi-turn dialogues. To address this problem, this paper introduces a novel Multi-Level Speaker Adaptive Network (ML-SAN), which, specifically, effectively addresses the challenge of speaker identity information confusion. ML-SAN does not simply assign a speaker's ID after recognition; instead, it employs a three-stage adaptive process: First, Input-level Calibration uses Feature-Level Linear Modulation (FiLM) to adjust the raw audio and visual features into a neutral space unrelated to the speaker. Then, Interaction-level Gating re-adjusts the trust level for each modality (e.g., voice or facial features) based on the speaker's identity information. Finally, Output-level Regularization maintains the consistency of speaker features in the latent space. Tests on the MELD and IEMOCAP datasets show that our model (ML-SAN) achieves better results, performs exceptionally well in handling challenging tail sentiment categories, and better addresses the diversity of speakers in real-world scenarios.
