A Statistical Framework for Detecting Emergent Narratives in Longitudinal Text Corpora

2026-02-24

AI Summary

This study proposes a statistical framework to identify emerging narratives in longitudinal textual corpora that reflect structural shifts in discourse, rather than superficial fluctuations in language use. By operationalizing “narrative emergence” as a sustained increase in the relative salience of latent topics, the method integrates topic modeling—based on Latent Dirichlet Allocation (LDA)—with time-series analysis and formal statistical inference. The approach is validated against external observable signals, such as the timing of Nobel Memorial Prizes in Economic Sciences. Applied to economics literature from 1970 to 2018, the framework successfully detects topics closely aligned with Nobel-recognized contributions, exhibiting significant upward trajectories that coincide with surges in citations and broader disciplinary recognition. This provides a statistically testable foundation for tracing structural semantic change in scholarly discourse over time.

Technology Category

• Natural Language Processing: Sentiment Analysis, Stylistic Analysis, and Argument Mining
• Game Theory and Economic Paradigms: Applications
• Data Mining & Knowledge Management: Semantic Web

Application Category

• Web Mining and Content Analysis: Topic discovery and tracking
• Economics, Online Markets and Human Computation: Sustainability of Web economics
• Semantics and Knowledge: Scalable techniques for the creation, curation, publication, maintenance, and consumption of large, Web-based, structured, reusable, knowledge graphs and ontologies

Abstract

Narratives about economic events and policies are widely recognised as influential drivers of economic and business behaviour. Yet the statistical identification of narrative emergence remains underdeveloped. Narratives evolve gradually, exhibit subtle shifts in content, and may exert influence disproportionate to their observable frequency, making it difficult to determine when observed changes reflect genuine structural shifts rather than routine variation in language use. We propose a statistical framework for detecting narrative emergence in longitudinal text corpora using Latent Dirichlet Allocation (LDA). We define emergence as a sustained increase in a topic's relative prominence over time and articulate a statistical framework for interpreting such trajectories, recognising that topic proportions are latent, model-estimated quantities. We illustrate the approach using a corpus of academic publications in economics spanning 1970-2018, where Nobel Prize-recognised contributions serve as externally observable signals of influential narratives. Topics associated with these contributions display sustained increases in estimated prevalence that coincide with periods of heightened citation activity and broader disciplinary recognition. These findings indicate that model-based topic trajectories can reflect identifiable shifts in economic discourse and provide a statistically grounded basis for analysing thematic change in longitudinal textual data.

Problem

• emergent narratives
• longitudinal text corpora
• statistical detection
• topic emergence
• economic discourse

Innovation

• narrative emergence
• Latent Dirichlet Allocation
• longitudinal text analysis
• topic modeling
• statistical framework