Can LLMs Learn Macroeconomic Narratives from Social Media?

2024-06-17

arXiv.org

AI Summary

This study investigates whether economic narratives from social media enhance macroeconomic forecasting accuracy, empirically testing the narrative economics hypothesis. Methodologically, we construct the first large-scale economic tweet dataset from X (formerly Twitter) and propose an LLM-driven framework for automated narrative extraction, clustering, and embedding—enabling structured representation of economic sentiment and themes. These narrative embeddings are integrated into time-series forecasting models to assess their marginal predictive power for key macroeconomic indicators, including GDP growth and CPI. Results demonstrate statistically significant improvements in out-of-sample forecast accuracy. Our contributions are threefold: (1) the first systematic empirical validation of narratives’ incremental value in macroeconomic prediction; (2) identification and analysis of core challenges in narrative data modeling—namely timeliness, noise robustness, and semantic drift; and (3) open-sourcing of a high-quality economic narrative dataset and a reproducible NLP pipeline, advancing quantitative research and practical applications in narrative economics.

Technology Category

• Natural Language Processing: Fact-Checking / Misinformation Detection (NLP Focus)
• Machine Learning: Time-Series/Data Streams
• Game Theory and Economic Paradigms: Behavioral Game Theory

Application Category

• Economics, Online Markets and Human Computation: Humans versus LLMs for data annotation and labeling
• Web Mining and Content Analysis: Large pretrained models with web data
• Social Networks and Social Media: Computational social science

Abstract

This study empirically tests the $ extit{Narrative Economics}$ hypothesis, which posits that narratives (ideas that are spread virally and affect public beliefs) can influence economic fluctuations. We introduce two curated datasets containing posts from X (formerly Twitter) which capture economy-related narratives (Data will be shared upon paper acceptance). Employing Natural Language Processing (NLP) methods, we extract and summarize narratives from the tweets. We test their predictive power for $ extit{macroeconomic}$ forecasting by incorporating the tweets' or the extracted narratives' representations in downstream financial prediction tasks. Our work highlights the challenges in improving macroeconomic models with narrative data, paving the way for the research community to realistically address this important challenge. From a scientific perspective, our investigation offers valuable insights and NLP tools for narrative extraction and summarization using Large Language Models (LLMs), contributing to future research on the role of narratives in economics.

Problem

• LLMs learn macroeconomic narratives
• NLP extracts economic narratives
• Narratives improve macroeconomic forecasting

Innovation

• LLMs for narrative extraction
• NLP for macroeconomic forecasting
• Social media data analysis