Search papers, labs, and topics across Lattice.
This survey paper examines the application of both Large Language Models (LLMs) and Multimodal Large Language Models (MLLMs) to the task of multimodal emotion recognition and reasoning, a rapidly growing area within AI for Science. It addresses the gap in systematic reviews by consolidating recent developments in model architectures, datasets, and performance benchmarks within this intersection. The survey also identifies key challenges and proposes future research directions for advancing the field.
LLMs are making strides in understanding emotions, but this survey reveals the current state-of-the-art in multimodal emotion recognition and reasoning, highlighting gaps and opportunities for improvement.
In recent years, large language models (LLMs) have driven major advances in language understanding, marking a significant step toward artificial general intelligence (AGI). With increasing demands for higher-level semantics and cross-modal fusion, multimodal large language models (MLLMs) have emerged, integrating diverse information sources (e.g., text, vision, and audio) to enhance modeling and reasoning in complex scenarios. In AI for Science, multimodal emotion recognition and reasoning has become a rapidly growing frontier. While LLMs and MLLMs have achieved notable progress in this area, the field still lacks a systematic review that consolidates recent developments. To address this gap, this paper provides a comprehensive survey of LLMs and MLLMs for emotion recognition and reasoning, covering model architectures, datasets, and performance benchmarks. We further highlight key challenges and outline future research directions, aiming to offer researchers both an authoritative reference and practical insights for advancing this domain. To the best of our knowledge, this paper is the first attempt to comprehensively survey the intersection of MLLMs with multimodal emotion recognition and reasoning. The summary of existing methods mentioned is in our Github: \href{https://github.com/yuntaoshou/Awesome-Emotion-Reasoning}{https://github.com/yuntaoshou/Awesome-Emotion-Reasoning}.
