Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper surveys the evolution of Emotion Recognition in Video (ERV) systems, tracing the shift from handcrafted features and task-specific deep learning models to transformer-based vision-language models and multimodal large language models (MLLMs). It analyzes multimodal fusion strategies, dataset characteristics, and evaluation protocols, while highlighting limitations related to robustness, bias, and annotation quality. The review compares task-specific models with foundation model approaches, clarifying their strengths and weaknesses for different application contexts, and outlines future research directions for robust and efficient ERV systems.
Task-specific emotion recognition models still hold their own against MLLMs in resource-constrained scenarios, challenging the assumption that bigger is always better.
Emotion recognition in video (ERV) aims to infer human affect from visual, audio, and contextual signals and is increasingly important for interactive and intelligent systems. Over the past decade, ERV has evolved from handcrafted features and task-specific deep learning models toward transformer-based vision–language models and multimodal large language models (MLLMs). This review surveys this evolution, with an emphasis on engineering considerations relevant to real-world deployment. We analyze multimodal fusion strategies, dataset characteristics, and evaluation protocols, highlighting limitations in robustness, bias, and annotation quality under unconstrained conditions. Emerging MLLM-based approaches are examined in terms of performance, reasoning capability, computational cost, and interaction potential. By comparing task-specific models with foundation model approaches, we clarify their respective strengths for resource-constrained versus context-aware applications. Finally, we outline practical research directions toward building robust, efficient, and deployable ERV systems for applied scenarios such as assistive technologies and human–AI interaction.