Problem
Research questions and friction points this paper is trying to address.
Large Language Models
Fake News Detection
Social Media
Explore the Potential of LLMs in Misinformation Detection: An Empirical Study
🤖 AI Summary
This study systematically evaluates large language models (LLMs) for social media rumor detection, focusing on their dual capabilities in content semantic understanding and propagation structure modeling. We construct zero-shot and few-shot detectors using mainstream open- and closed-source LLMs (e.g., GPT, Llama) across eight benchmark datasets, integrating prompt engineering and model ensembling. Results show that LLMs achieve performance on par with task-specific models for pure textual rumor classification, yet significantly underperform in modeling propagation structures—such as retweet chains and user interactions. To address this gap, we propose a novel differentiable LLM-augmented module that is jointly fine-tuned with existing propagation-based detectors, yielding substantial performance gains. This work provides the first comprehensive characterization of LLMs’ capability boundary in rumor detection—“strong semantics, weak structure”—and empirically validates their practical utility as trainable enhancement components for structural propagation modeling.
📝 Abstract
Large Language Models (LLMs) have garnered significant attention for their powerful ability in natural language understanding and reasoning. In this paper, we present a comprehensive empirical study to explore the performance of LLMs on misinformation detection tasks. This study stands as the pioneering investigation into the understanding capabilities of multiple LLMs regarding both content and propagation across social media platforms. Our empirical studies on eight misinformation detection datasets show that LLM-based detectors can achieve comparable performance in text-based misinformation detection but exhibit notably constrained capabilities in comprehending propagation structure compared to existing models in propagation-based misinformation detection. Our experiments further demonstrate that LLMs exhibit great potential to enhance existing misinformation detection models. These findings highlight the potential ability of LLMs to detect misinformation.
Innovation
Methods, ideas, or system contributions that make the work stand out.
Large Language Models
Fake News Detection
Message Propagation Analysis
🔎 Similar Papers
No similar papers found.
