Trust but Verify: Introducing DAVinCI -- A Framework for Dual Attribution and Verification in Claim Inference for Language Models

2026-04-22

AI Summary

This work addresses the critical challenge of factual unreliability and hallucination in large language models (LLMs) when deployed in high-stakes domains such as healthcare and law. To enhance trustworthiness, the authors propose DAVinCI, a novel framework that tightly integrates claim attribution with verification for the first time. DAVinCI first attributes generated claims to both internal model knowledge and external evidence sources, then performs end-to-end verification through textual entailment reasoning coupled with confidence calibration. The framework enables auditable and traceable reasoning while supporting modular integration into existing LLM pipelines. Evaluated on benchmarks including FEVER and CLIMATE-FEVER, DAVinCI achieves significant improvements—ranging from 5% to 20%—in classification accuracy and attribution metrics (precision, recall, and F1), thereby substantially enhancing the factual reliability and interpretability of model outputs.

Technology Category

• Natural Language Processing: Fact-Checking / Misinformation Detection (NLP Focus)
• Machine Learning: Large Multimodal Models (LMMs)
• Computer Vision: Large Vision Models

Application Category

• Search and Retrieval-Augmented AI: Large language models for search
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

Large Language Models (LLMs) have demonstrated remarkable fluency and versatility across a wide range of NLP tasks, yet they remain prone to factual inaccuracies and hallucinations. This limitation poses significant risks in high-stakes domains such as healthcare, law, and scientific communication, where trust and verifiability are paramount. In this paper, we introduce DAVinCI - a Dual Attribution and Verification framework designed to enhance the factual reliability and interpretability of LLM outputs. DAVinCI operates in two stages: (i) it attributes generated claims to internal model components and external sources; (ii) it verifies each claim using entailment-based reasoning and confidence calibration. We evaluate DAVinCI across multiple datasets, including FEVER and CLIMATE-FEVER, and compare its performance against standard verification-only baselines. Our results show that DAVinCI significantly improves classification accuracy, attribution precision, recall, and F1-score by 5-20%. Through an extensive ablation study, we isolate the contributions of evidence span selection, recalibration thresholds, and retrieval quality. We also release a modular DAVinCI implementation that can be integrated into existing LLM pipelines. By bridging attribution and verification, DAVinCI offers a scalable path to auditable, trustworthy AI systems. This work contributes to the growing effort to make LLMs not only powerful but also accountable.

Problem

• factual inaccuracies
• hallucinations
• trustworthiness
• verification
• attribution

Innovation

• Dual Attribution
• Claim Verification
• Entailment-based Reasoning
• Confidence Calibration
• Trustworthy LLMs