Retrieve-Refine-Calibrate: A Framework for Complex Claim Fact-Checking

2026-01-23

AI Summary

This work proposes a retrieval–refinement–calibration (RRC) framework based on large language models to address the challenge of verifying complex factual claims, where existing methods often introduce irrelevant entities or evidence, thereby compromising accuracy. The RRC framework first identifies key entities within the claim and retrieves pertinent evidence, then refines this evidence according to the claim’s semantic content to eliminate noise, and finally calibrates predictions with low confidence. By eschewing conventional decomposition paradigms and instead integrating evidence refinement with prediction calibration, the approach effectively suppresses distracting information. Experimental results on the HOVER and FEVEROUS-S datasets demonstrate that RRC significantly outperforms current baselines, underscoring its effectiveness and novelty in enhancing the accuracy of complex claim verification.

Technology Category

• Natural Language Processing: Fact-Checking / Misinformation Detection (NLP Focus)
• Knowledge Representation and Reasoning: Computational Complexity of Reasoning
• Machine Learning: Calibration & Uncertainty Quantification

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

Fact-checking aims to verify the truthfulness of a claim based on the retrieved evidence. Existing methods typically follow a decomposition paradigm, in which a claim is broken down into sub-claims that are individually verified. However, the decomposition paradigm may introduce noise to the verification process due to irrelevant entities or evidence, ultimately degrading verification accuracy. To address this problem, we propose a Retrieve-Refine-Calibrate (RRC) framework based on large language models (LLMs). Specifically, the framework first identifies the entities mentioned in the claim and retrieves evidence relevant to them. Then, it refines the retrieved evidence based on the claim to reduce irrelevant information. Finally, it calibrates the verification process by re-evaluating low-confidence predictions. Experiments on two popular fact-checking datasets (HOVER and FEVEROUS-S) demonstrate that our framework achieves superior performance compared with competitive baselines.

Problem

• fact-checking
• decomposition paradigm
• verification accuracy
• irrelevant evidence
• noise

Innovation

• Retrieve-Refine-Calibrate
• fact-checking
• large language models
• evidence refinement
• verification calibration