The Warrant Gap: Claim-Conditioned Re-scoring for Fact-Checking

2026-06-23

AI Summary

This work addresses the “warrant gap” in fact-checking with large language models—where supportive verdicts are often issued without sufficient evidential grounding. To bridge this gap, the authors propose SIFT, a method that preserves full contextual information through claim-conditional re-ranking and automatically validates whether retrieved evidence genuinely entails the claim using a natural language inference (NLI)-based Warrant Support Precision (WSP) metric. By integrating structured evidence decomposition, conditional re-scoring, and automated warrant plausibility assessment, SIFT substantially outperforms standard prompting approaches across four benchmarks, including FEVER and SciFact, achieving accuracy gains of up to 27.6 points and attaining a WSP AUC of 0.92 with a precision of 0.98.

Abstract

Fact-checking systems built on LLMs achieve high verdict accuracy on standard benchmarks, yet routinely output Supports labels whose cited evidence does not license the claim. Structured decomposition is the natural way to inspect those warrants, but rigid extraction protocols strip the full-claim context that facets need. We introduce SIFT -- claim-conditioned re-scoring of extracted evidence spans against the full claim -- paired with WSP (Warranted Supports Proportion), an automatic NLI check that the cited warrant entails the claim. We evaluate on FEVER, SciFact, 5PILS, and DP across four open-source backbones. SIFT recovers accuracy on cells where naive decomposition costs up to 27.6 points, while raising WSP above direct prompting; WSP itself calibrates against human gold evidence at AUC 0.92 and precision 0.98.

Problem

• fact-checking
• warrant gap
• claim-evidence alignment
• natural language inference
• structured decomposition

Innovation

• claim-conditioned re-scoring
• warrant gap
• fact-checking
• natural language inference
• evidence calibration