Temporal reasoning for timeline summarisation in social media

2024-12-30

AI Summary

This study addresses the degradation of summarization quality in long-term social media event streams—particularly mental health news—caused by temporal interleaving of multiple events, mixed sentiment expressions, and redundant repetitions. To tackle this, we propose a temporal summarization method that enhances large language models’ (LLMs) temporal reasoning capabilities. Our key contributions are: (1) the construction of NarrativeReason, the first dataset explicitly designed for modeling intra-narrative temporal relations among multiple events; and (2) a novel knowledge distillation framework jointly optimizing temporal reasoning and timeline summarization, integrating temporal relation modeling, narrative structure-aware training, and LLM fine-tuning. Experiments on mental health news summarization demonstrate that our method significantly outperforms strong baselines, effectively mitigating event redundancy and sentiment confusion, with a 3.2-point improvement in ROUGE-L score.

Technology Category

• Natural Language Processing: Summarization
• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning
• Machine Learning: Large Multimodal Models (LMMs)

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

This paper explores whether enhancing temporal reasoning capabilities in Large Language Models (LLMs) can improve the quality of timeline summarization, the task of summarising long texts containing sequences of events, particularly social media threads . We introduce extit{NarrativeReason}, a novel dataset focused on temporal relationships among sequential events within narratives, distinguishing it from existing temporal reasoning datasets that primarily address pair-wise event relationships. Our approach then combines temporal reasoning with timeline summarization through a knowledge distillation framework, where we first fine-tune a teacher model on temporal reasoning tasks and then distill this knowledge into a student model while simultaneously training it for the task of timeline summarization. Experimental results demonstrate that our model achieves superior performance on mental health-related timeline summarization tasks, which involve long social media threads with repetitions of events and a mix of emotions, highlighting the importance of leveraging temporal reasoning to improve timeline summarisation.

Problem

• Complex Event Sequences
• Sentiment Analysis
• Social Media Summarization

Innovation

• Temporal Reasoning
• Knowledge Distillation
• NarrativeSummarization