Extracting Affect Aggregates from Longitudinal Social Media Data with Temporal Adapters for Large Language Models

2024-09-26

AI Summary

This study investigates the longitudinal evolution of public sentiment and attitudes in the UK during the early phase of the COVID-19 pandemic (2020). To address the challenge of unsupervised, time-resolved sentiment analysis on social media time-series data, we propose a novel framework leveraging time-aligned large language models (LLMs): it introduces *Temporal Adapters*—the first such adaptation for LLMs—enabling dynamic sentiment modeling without labeled data or task-specific classifier pretraining. Built upon Llama-3-8B, the framework integrates standardized psychological sentiment scale mappings and employs multi-seed/multi-prompt robustness validation. Results demonstrate statistically significant positive correlations (p < 0.01) between model-derived sentiment metrics and representative UK survey data; moreover, outputs exhibit high consistency across random seeds and prompt variants, matching supervised classification performance. This work establishes a scalable, low-resource paradigm for long-term, fine-grained sentiment tracking of public discourse.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: (Large) Language Models
• Application Domains: Humanities & Computational Social Science

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Web Mining and Content Analysis: Large pretrained models with web data
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

This paper proposes temporally aligned Large Language Models (LLMs) as a tool for longitudinal analysis of social media data. We fine-tune Temporal Adapters for Llama 3 8B on full timelines from a panel of British Twitter users, and extract longitudinal aggregates of emotions and attitudes with established questionnaires. We focus our analysis on the beginning of the COVID-19 pandemic that had a strong impact on public opinion and collective emotions. We validate our estimates against representative British survey data and find strong positive, significant correlations for several collective emotions. The obtained estimates are robust across multiple training seeds and prompt formulations, and in line with collective emotions extracted using a traditional classification model trained on labeled data. We demonstrate the flexibility of our method on questions of public opinion for which no pre-trained classifier is available. Our work extends the analysis of affect in LLMs to a longitudinal setting through Temporal Adapters. It enables flexible, new approaches towards the longitudinal analysis of social media data.

Problem

• Social Media
• Public Sentiment
• COVID-19

Innovation

• Large Language Model Analysis
• Social Media Sentiment Dynamics
• COVID-19 Emotional Fluctuations