Explainable Biomedical Claim Verification with Large Language Models

2025-02-28

AI Summary

To address insufficient transparency and interpretability in biomedical claim verification, this paper proposes the first end-to-end interpretable verification framework integrating task-adaptive LLM-based natural language inference (NLI), token-level SHAP attribution, and user-guided narrative rationale generation. The system supports literature retrieval, multi-LLM collaborative NLI classification, evidence synthesis, consensus decision-making, and user intervention in the reasoning process. Its key innovation lies in the deep integration of SHAP-based interpretability analysis into the LLM-NLI pipeline—enabling simultaneous fine-grained attribution and natural-language rationale generation. Evaluated on multiple clinical claim verification tasks, the framework achieves expert-level inter-rater agreement (≥92%), substantially enhancing trustworthiness, traceability, and accountability in human-AI collaborative decision-making. It provides a secure, reliable, and interpretable AI foundation for clinical decision support, public health policy formulation, and biomedical research.

Technology Category

• Natural Language Processing: Fact-Checking / Misinformation Detection (NLP Focus)
• Philosophy and Ethics of AI: Accountability, Interpretability & Explainability
• Knowledge Representation and Reasoning: Diagnosis and Abductive Reasoning

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Economics, Online Markets and Human Computation: Humans versus LLMs for data annotation and labeling
• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization

Abstract

Verification of biomedical claims is critical for healthcare decision-making, public health policy and scientific research. We present an interactive biomedical claim verification system by integrating LLMs, transparent model explanations, and user-guided justification. In the system, users first retrieve relevant scientific studies from a persistent medical literature corpus and explore how different LLMs perform natural language inference (NLI) within task-adaptive reasoning framework to classify each study as"Support,""Contradict,"or"Not Enough Information"regarding the claim. Users can examine the model's reasoning process with additional insights provided by SHAP values that highlight word-level contributions to the final result. This combination enables a more transparent and interpretable evaluation of the model's decision-making process. A summary stage allows users to consolidate the results by selecting a result with narrative justification generated by LLMs. As a result, a consensus-based final decision is summarized for each retrieved study, aiming safe and accountable AI-assisted decision-making in biomedical contexts. We aim to integrate this explainable verification system as a component within a broader evidence synthesis framework to support human-AI collaboration.

Problem

• Develops interactive system for biomedical claim verification
• Integrates LLMs with transparent explanations for decision-making
• Supports safe AI-assisted decisions in healthcare contexts

Innovation

• Integrates LLMs for biomedical claim verification
• Uses SHAP values for transparent model explanations
• Enables user-guided justification for decision-making