Search papers, labs, and topics across Lattice.
This paper introduces LG-GER, a novel language-guided distillation framework for group emotion recognition (GER) that integrates multimodal evidence from images without relying on traditional detector-driven pipelines. By utilizing a multimodal large language model (MLLM) to generate spatially grounded evidence, the approach distills this information into a single vision-language model (VLM) through four complementary losses, enhancing the model's ability to infer collective emotional states. The results demonstrate that LG-GER outperforms existing state-of-the-art methods on benchmark datasets while simplifying the inference process, making it suitable for real-time applications.
LG-GER achieves superior group emotion recognition without the need for complex multi-stream processing or detectors, streamlining deployment in resource-constrained environments.
Inferring the collective emotional state of a group of people from a single image, a task known as group emotion recognition (GER), requires integrating spatially distributed cues such as faces, poses, interactions, and scene context. Current methods rely on detector-driven multi-stream pipelines. These are trained with only image-level supervision that lacks guidance on which regions matter or how strongly each contributes. We propose LG-GER, a language-guided distillation framework that uses a multimodal large language model (MLLM) to generate dense, spatially grounded evidence, i.e., bounding boxes paired with emotion signals and confidence scores, for the training images. This structured evidence is distilled into a single vision-language model (VLM) backbone through four complementary losses: classification, region-text grounding, spatial emotion, and spatial confidence regression. At inference, LG-GER requires no detectors, no MLLM, and no multi-stream fusion, making GER practical for real-time and resource-constrained deployment. LG-GER has been evaluated on two benchmark GER datasets (GroupEmoW and GAF~3.0) and achieves competitive or superior results compared to state-of-the-art methods that require detection and multi-stream processing at inference.
