Towards Real-Time Fake News Detection under Evidence Scarcity

2025-10-13

AI Summary

To address the challenge of real-time fake news detection under scarce emerging-event evidence, this paper proposes EASE, an Evaluation-Aware Expert Selection framework. EASE dynamically adapts to evidence sufficiency by orchestrating three orthogonal evaluation modules—evidence-driven, reasoning-driven, and sentiment-fallback—and employs interpretable instruction tuning to generate high-quality pseudo-labels for evaluation-aware decision-making. The method synergistically integrates large language models’ world knowledge, multi-module ensemble, and interpretable reasoning to significantly enhance generalization. EASE achieves state-of-the-art performance across multiple benchmarks, demonstrating exceptional robustness in emerging-news scenarios. To foster reproducible research, we introduce RealTimeNews-25—a new benchmark specifically designed for real-time fake news detection—and fully open-source both code and datasets.

Technology Category

• Natural Language Processing: Fact-Checking / Misinformation Detection (NLP Focus)
• Application Domains: Misinformation & Fake News
• Machine Learning: Evaluation and Analysis

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Economics, Online Markets and Human Computation: Trust and reliance of crowd workers and data experts on GenAI

Abstract

Fake news detection becomes particularly challenging in real-time scenarios, where emerging events often lack sufficient supporting evidence. Existing approaches often rely heavily on external evidence and therefore struggle to generalize under evidence scarcity. To address this issue, we propose Evaluation-Aware Selection of Experts (EASE), a novel framework for real-time fake news detection that dynamically adapts its decision-making process according to the assessed sufficiency of available evidence. EASE introduces a sequential evaluation mechanism comprising three independent perspectives: (1) Evidence-based evaluation, which assesses evidence and incorporates it into decision-making only when the evidence is sufficiently supportive; (2) Reasoning-based evaluation, which leverages the world knowledge of large language models (LLMs) and applies them only when their reliability is adequately established; and (3) Sentiment-based fallback, which integrates sentiment cues when neither evidence nor reasoning is reliable. To enhance the accuracy of evaluation processes, EASE employs instruction tuning with pseudo labels to guide each evaluator in justifying its perspective-specific knowledge through interpretable reasoning. Furthermore, the expert modules integrate the evaluators' justified assessments with the news content to enable evaluation-aware decision-making, thereby enhancing overall detection accuracy. Moreover, we introduce RealTimeNews-25, a new benchmark comprising recent news for evaluating model generalization on emerging news with limited evidence. Extensive experiments demonstrate that EASE not only achieves state-of-the-art performance across multiple benchmarks, but also significantly improves generalization to real-time news. The code and dataset are available: https://github.com/wgyhhhh/EASE.

Problem

• Detecting fake news in real-time with limited evidence
• Adapting decision-making based on evidence sufficiency assessment
• Improving generalization for emerging news lacking sufficient support

Innovation

• Dynamic expert selection based on evidence sufficiency
• Sequential evaluation using evidence, reasoning, and sentiment
• Instruction tuning with pseudo labels for interpretable reasoning