When Claims Evolve: Evaluating and Enhancing the Robustness of Embedding Models Against Misinformation Edits

2025-03-05

AI Summary

Existing embedded fact-checking retrieval methods exhibit insufficient robustness when verifying online misinformation that has been edited and re-posted. Method: We propose the first structured perturbation generation framework specifically designed for misleading edits, establishing a taxonomy of six realistic edit types. Our approach integrates training- and inference-time robustness enhancements, combining LLM-distilled embeddings, edit-aware fine-tuning, adversarial training, and a retrieval-reranking collaborative architecture. Contribution/Results: Evaluated across multiple stages, our method improves in-domain retrieval accuracy by 17 percentage points and cross-domain generalization by 10 percentage points. It significantly strengthens mainstream embedding models’ ability to match edited claims with their original factual evidence, thereby providing critical technical support for real-time, robust fact-checking systems.

Technology Category

• Natural Language Processing: Fact-Checking / Misinformation Detection (NLP Focus)
• Machine Learning: Adversarial Learning & Robustness
• Computer Vision: Adversarial Attacks & Robustness

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods

Abstract

Online misinformation remains a critical challenge, and fact-checkers increasingly rely on embedding-based methods to retrieve relevant fact-checks. Yet, when debunked claims reappear in edited forms, the performance of these methods is unclear. In this work, we introduce a taxonomy of six common real-world misinformation edits and propose a perturbation framework that generates valid, natural claim variations. Our multi-stage retrieval evaluation reveals that standard embedding models struggle with user-introduced edits, while LLM-distilled embeddings offer improved robustness at a higher computational cost. Although a strong reranker helps mitigate some issues, it cannot fully compensate for first-stage retrieval gaps. Addressing these retrieval gaps, our train- and inference-time mitigation approaches enhance in-domain robustness by up to 17 percentage points and boost out-of-domain generalization by 10 percentage points over baseline models. Overall, our findings provide practical improvements to claim-matching systems, enabling more reliable fact-checking of evolving misinformation.

Problem

• Evaluating embedding models' robustness against misinformation edits.
• Proposing a framework to generate natural claim variations.
• Enhancing claim-matching systems for reliable fact-checking.

Innovation

• Proposes taxonomy for misinformation edits
• Introduces perturbation framework for claim variations
• Enhances robustness with train-time mitigation approaches