MisSpans: Fine-Grained False Span Identification in Cross-Domain Fake News
🤖 AI Summary
This work addresses the limitations of existing fake news detection methods, which predominantly rely on coarse-grained binary labels and struggle to identify fine-grained misinformation within individual sentences or provide interpretability. To bridge this gap, the authors introduce MisSpans, the first cross-domain benchmark for fine-grained misinformation annotation, comprising paired real and fake news stories. The benchmark defines three tasks: misinformation span localization, error type classification, and span-based explanation generation. A high-quality, multi-domain dataset is constructed through standardized annotation guidelines, expert labeling, and rigorous consistency checks. Evaluation across 15 large language models reveals significant limitations in current models’ ability to detect fine-grained misinformation, with performance jointly influenced by model scale, reasoning capabilities, and domain-specific characteristics.
Application Category
📝 Abstract
Online misinformation is increasingly pervasive, yet most existing benchmarks and methods evaluate veracity at the level of whole claims or paragraphs using coarse binary labels, obscuring how true and false details often co-exist within single sentences. These simplifications also limit interpretability: global explanations cannot identify which specific segments are misleading or differentiate how a detail is false (e.g., distorted vs. fabricated). To address these gaps, we introduce MisSpans, the first multi-domain, human-annotated benchmark for span-level misinformation detection and analysis, consisting of paired real and fake news stories. MisSpans defines three complementary tasks: MisSpansIdentity for pinpointing false spans within sentences, MisSpansType for categorising false spans by misinformation type, and MisSpansExplanation for providing rationales grounded in identified spans. Together, these tasks enable fine-grained localisation, nuanced characterisation beyond true/false and actionable explanations. Expert annotators were guided by standardised guidelines and consistency checks, leading to high inter-annotator agreement. We evaluate 15 representative LLMs, including reasoning-enhanced and non-reasoning variants, under zero-shot and one-shot settings. Results reveal the challenging nature of fine-grained misinformation identification and analysis, and highlight the need for a deeper understanding of how performance may be influenced by multiple interacting factors, including model size and reasoning capabilities, along with domain-specific textual features. This project will be available at https://github.com/lzw108/MisSpans.
Problem
Research questions and friction points this paper is trying to address.
fake news
misinformation detection
fine-grained analysis
span-level identification
interpretability
Innovation
Methods, ideas, or system contributions that make the work stand out.
fine-grained misinformation detection
span-level annotation
misinformation typology
explainable AI
cross-domain fake news
🔎 Similar Papers
No similar papers found.
