Problem
Research questions and friction points this paper is trying to address.
emotion analysis
social media posts
image modality
triggering events
emotion reconstruction
Emotion Experience, Expression, and Perception: Emotion Analysis on Multimodal Social Media Posts
🤖 AI Summary
研究通过构建Mult2EMo数据集，分析社交媒体帖子中的文本和图像如何共同表达情绪，以及读者理解这些情绪的能力，强调了触发事件在情绪理解中的重要性。
📝 Abstract
Emotions are an essential aspect of human communication, particularly on social media, where authors frequently combine text and images to convey their emotions. Yet prior work on emotion analysis of social media posts has overlooked two important aspects in regard to measuring how well readers can reconstruct the authors' intent: (1)~the image modality, with most work focusing solely on text, and (2)~the real-world events that trigger the expressed emotions, and their relationship to the post content. We therefore study the relation between (a) the author's experience of the event that caused them to write a social media post and (b) the content of the post, with a focus on readers' capability to reconstruct that emotion expression. To do that, we introduce the Multimodal Multi-Emotion-Model dataset Mult2EMo, created by collecting annotations from both authors and readers on the posts and their triggering events. We find that reconstruction is possible but challenging for both human readers and computational models. We show that understanding the triggering event is crucial for accurate reconstruction, and that reconstruction is particularly challenging when posts rely heavily on the image to express emotion.
Innovation
Methods, ideas, or system contributions that make the work stand out.
Multimodal
Emotion Analysis
Triggering Event
Social Media Posts
🔎 Similar Papers
No similar papers found.
