DeepSciVerify: Verifying Scientific Claim--Citation Alignment via LLM-Driven Evidence Escalation

2026-05-26

AI Summary

This work addresses the inconsistency between claims and supporting evidence in scientific reports generated by large language models by proposing a two-stage verification approach. The method first validates claims against abstracts and, for uncertain cases, selectively retrieves and analyzes relevant full-text passages. It innovatively integrates abstract-level reasoning with an on-demand full-text evidence escalation mechanism, leveraging the complementary behaviors of different large language models under uncertainty to enhance accuracy without sacrificing efficiency. Evaluated on the SCitance benchmark, the approach achieves a Micro-F1 score of 86.7, outperforming strong abstract-only baselines by 4.5 points, and correctly resolves 67% of instances without requiring full-text retrieval.

Abstract

Misalignment between claims and their cited evidence is a common failure mode in reports generated by large language models, limiting their reliability in scientific and other high-stakes settings. We present DeepSciVerify, a two-stage pipeline for scientific claim-citation verification that combines abstract-level reasoning with selective escalation to passage-level evidence. The system first verifies claims using the abstract and defers uncertain cases, retrieving and analyzing full-text passages only when necessary. This design leverages complementary behaviors across LLMs, as some models are more conservative while others are more decisive under uncertainty. On the SCitance benchmark, DeepSciVerify achieves 86.7 Micro-F1, outperforming strong abstract-only baselines by +4.5 points while resolving 67% of instances without full-text retrieval. These results suggest that selective evidence escalation improves both accuracy and efficiency in claim-citation verification.

Problem

• claim-citation alignment
• scientific verification
• evidence escalation
• large language models
• misalignment

Innovation

• claim-citation verification
• evidence escalation
• LLM-driven reasoning
• scientific fact-checking
• two-stage verification