Emotion Alignment: Discovering the Gap Between Social Media and Real-World Sentiments in Persian Tweets and Images

2025-04-14

AI Summary

This study investigates alignment discrepancies between Persian users’ social media expressions (tweets and images) on X and their real-world affective states. Methodologically, we construct the first cross-modal affective alignment framework for Persian-speaking communities, grounded in data from 105 participants, 393 peer/family affective assessments, 8,300+ tweets, and 2,000 images. We propose a novel “real-virtual affective distance” metric and pioneer tri-source affective calibration—integrating textual, visual, and third-party behavioral reports. Our approach employs a multimodal Transformer architecture combining BERT variants and Vision Transformers (ViT), augmented with a social-relationship-driven annotation scheme and a statistical significance testing framework. Results reveal a 75.88% alignment rate between tweets and ground-truth affect, versus only 28.67% for images—a highly significant difference (p < 0.01)—demonstrating systematic affective misalignment in online expression.

Technology Category

• Cognitive Modeling & Cognitive Systems: Affective Computing
• Computer Vision: Multi-modal Vision
• Application Domains: Humanities & Computational Social Science

Application Category

• Web Mining and Content Analysis: Mining multimedia, multimodal, multilingual, cross-lingual Web data
• User Modeling, Personalization and Recommendation: Metrics for user behavior and evaluating success
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics

Abstract

In contemporary society, widespread social media usage is evident in people's daily lives. Nevertheless, disparities in emotional expressions between the real world and online platforms can manifest. We comprehensively analyzed Persian community on X to explore this phenomenon. An innovative pipeline was designed to measure the similarity between emotions in the real world compared to social media. Accordingly, recent tweets and images of participants were gathered and analyzed using Transformers-based text and image sentiment analysis modules. Each participant's friends also provided insights into the their real-world emotions. A distance criterion was used to compare real-world feelings with virtual experiences. Our study encompassed N=105 participants, 393 friends who contributed their perspectives, over 8,300 collected tweets, and 2,000 media images. Results indicated a 28.67% similarity between images and real-world emotions, while tweets exhibited a 75.88% alignment with real-world feelings. Additionally, the statistical significance confirmed that the observed disparities in sentiment proportions.

Problem

• Measure emotion gap between Persian social media and real life
• Compare sentiment similarity in tweets vs real-world feelings
• Analyze text and image emotions using Transformer models

Innovation

• Transformers-based sentiment analysis for text and images
• Distance criterion to compare real and virtual emotions
• Comprehensive dataset with tweets, images, and friend insights