Video2Reaction: Training Foundation Video Models to Predict Audience Reaction

2026-09-01

AI Summary

研究通过创建Video2Reaction数据集，利用观众社交媒体评论预测对电影片段的情绪反应，并使用LoRA微调的视觉-语言模型来解决情绪预测问题。

Abstract

We introduce Video2Reaction, a multimodal dataset that maps short movie segments to the induced emotional reactions of viewers in the wild, as expressed through social media comments. Video2Reaction captures the natural diversity of emotional responses by aggregating reactions from online comments at scale, modeling labels as distributions over categorical emotions to better reflect the subjective and ambiguous nature of emotional perception. We benchmark two vision-language models (VLMs) finetuned with LoRA, showing that VLMs learn effectively from Video2Reaction and outperform specialized baselines on dominant reaction prediction. We further demonstrate that VLMs pre-finetuned on Video2Reaction transfer effectively to VCE, another induced emotion dataset with a different taxonomy and video domain. Notably, LLaVA-NeXT-Video-7B pre-finetuned on Video2Reaction and adapted on only 1% of VCE training data achieves a top-3 accuracy of 0.682, on par with the best reported VCE performance trained on the full dataset. The dataset is available at https://huggingface.co/datasets/infofusionlab/Video2Reaction

Problem

• Video2Reaction
• emotional reactions
• social media comments
• multimodal dataset
• vision-language models

Innovation

• multimodal dataset
• emotional reactions
• LoRA fine-tuning
• VLMs
• cross-dataset transfer