LG-GER: Language-Guided Group Emotion Recognition via Multimodal Evidence Distillation

2026-08-24

AI Summary

为解决群体情绪识别中缺乏区域指导的问题，提出LG-GER方法，利用多模态大语言模型生成空间证据并通过四类损失函数蒸馏至视觉-语言模型。

Abstract

Inferring the collective emotional state of a group of people from a single image, a task known as group emotion recognition (GER), requires integrating spatially distributed cues such as faces, poses, interactions, and scene context. Current methods rely on detector-driven multi-stream pipelines. These are trained with only image-level supervision that lacks guidance on which regions matter or how strongly each contributes. We propose LG-GER, a language-guided distillation framework that uses a multimodal large language model (MLLM) to generate dense, spatially grounded evidence, i.e., bounding boxes paired with emotion signals and confidence scores, for the training images. This structured evidence is distilled into a single vision-language model (VLM) backbone through four complementary losses: classification, region-text grounding, spatial emotion, and spatial confidence regression. At inference, LG-GER requires no detectors, no MLLM, and no multi-stream fusion, making GER practical for real-time and resource-constrained deployment. LG-GER has been evaluated on two benchmark GER datasets (GroupEmoW and GAF~3.0) and achieves competitive or superior results compared to state-of-the-art methods that require detection and multi-stream processing at inference.

Problem

• group emotion recognition
• detector-driven
• image-level supervision

Innovation

• Language-Guided Distillation
• Multimodal Large Language Model (MLLM)
• Vision-Language Model (VLM)
• Dense Spatial Evidence