Search papers, labs, and topics across Lattice.
This paper introduces EmoAgent-R1, a Reinforcement Learning-based framework that enhances multimodal emotion recognition by employing dynamic agent specialization to adaptively perceive complex emotions from multimodal inputs. By utilizing a cold start strategy for initial training and a two-step workflow for agent selection and specialization, EmoAgent-R1 significantly improves the model's reasoning and generalization capabilities. Experimental results on emotion recognition benchmarks show that EmoAgent-R1 outperforms existing methods, achieving better reasoning performance and optimization stability.
EmoAgent-R1 reveals that dynamic agent specialization can dramatically enhance emotion reasoning in multimodal contexts, outperforming traditional fixed-prompt approaches.
Multimodal large language models (MLLMs) have achieved impressive performance in multimodal emotion recognition (MER) tasks and lifted MER to a new level that is complex emotion understanding with advanced video understanding abilities and natural language description. However, existing MLLM-based methods often use a fixed prompt to perceive the emotions, ignoring the dynamicity and complexity of the emotion source in the multimodal inputs. To address these issues, we propose a novel Reinforcement Learning-based Dynamic Agent Specialization framework (\textbf{EmoAgent-R1}) to optimize the emotion recognition, reasoning, and generalization abilities of an MLLM with dynamic agent specialization based on reinforcement learning. Specifically, we first adopt a cold start strategy to endow an MLLM with preliminary emotion recognition, reasoning, and agent routing ability by training with synthetic answer-conditioned chain-of-thought data and agent routing data. Then, we further train the MLLM with reinforcement learning to perceive emotions in a two-step agentic workflow with agent selection and agent specialization. To effectively train EmoAgent-R1, we propose a novel Progressive Group-Relative Policy Optimization (P-GRPO) to combine group-based relative advantages with a PMI-inspired progressive token-level modulation to transform sparse rewards into fine-grained learning signals, mitigating the coarse-grained uniform credit assignment issue in GRPO. Extensive experiments on MER benchmarks demonstrate the superiority of our EmoAgent-R1 in stronger emotion reasoning performance and improved optimization stability.
