Reasoning Visual Language Model for Chest X-Ray Analysis

2025-10-27

AI Summary

Current vision-language models (VLMs) for chest X-ray analysis lack interpretability, hindering clinical audit and human-AI collaboration. To address this, we propose the first reasoning-first VLM framework tailored to thoracic radiology diagnosis. It explicitly models radiologists’ systematic differential diagnostic process via chain-of-thought prompting, generating traceable, uncertainty-aware reasoning paths with alternative hypotheses. Methodologically, the framework integrates high-fidelity visual encoding, two-stage supervised fine-tuning, and verifiability-guided reinforcement learning to jointly model multiple abnormalities. Experiments demonstrate competitive performance on multi-label classification tasks. Radiologist evaluations confirm that the generated reasoning trajectories significantly enhance diagnostic confidence, accelerate report generation, and enable error tracing and decision audit. This work establishes a novel paradigm for trustworthy AI-assisted diagnosis in medical imaging.

Technology Category

• Knowledge Representation and Reasoning: Diagnosis and Abductive Reasoning
• Computer Vision: Interpretability, Explainability, and Transparency
• Humans and AI: Explainable AI (XAI) for Human Understanding

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

Vision-language models (VLMs) have shown strong promise for medical image analysis, but most remain opaque, offering predictions without the transparent, stepwise reasoning clinicians rely on. We present a framework that brings chain-of-thought (CoT) reasoning to chest X-ray interpretation. Inspired by reasoning-first training paradigms, our approach is designed to learn how experts reason, not just what they conclude, by aligning intermediate steps with observable image evidence and radiology workflow. Beyond accuracy, the explicit reasoning traces support clinical auditability: they reveal why a conclusion was reached, which alternatives were considered, and where uncertainty remains, enabling quality assurance, error analysis, and safer human-AI collaboration. Our model couples high-fidelity visual encoding with a two-stage training recipe: a reasoning-style supervised fine-tuning (SFT) followed by reinforcement learning (RL) that uses verifiable rewards over a list of X-ray abnormalities. The model outputs reasoning that mirrors radiologists systematic thought process, uncertainty, and differential diagnosis. In out-of-distribution evaluation, the approach achieves competitive multi-label classification while improving interpretability. In a reader study with expert radiologists, full reasoning traces increased confidence, supported error auditing, and reduced time to finalize reports. We release code and the model NV-Reason-CXR-3B to support community progress toward trustworthy, explainable AI in chest radiography and other medical imaging tasks where reasoning quality is as critical as prediction quality.

Problem

• Enhances transparency in medical image analysis through reasoning
• Aligns AI reasoning steps with radiology workflow and evidence
• Supports clinical auditability and safer human-AI collaboration

Innovation

• Chain-of-thought reasoning for chest X-ray interpretation
• Two-stage training with SFT and verifiable RL rewards
• Explicit reasoning traces enabling clinical auditability