MedReason-R1: Learning to Reason for CT Diagnosis with Reinforcement Learning and Local Zoom

2025-10-22

AI Summary

General-purpose vision-language models (VLMs) underperform in medical image diagnosis due to scarce high-quality medical visual question answering (VQA) data and insufficient modeling of the clinically essential “coarse-to-fine” diagnostic reasoning process. Method: We introduce CT-RATE-VQA, a large-scale medical CT VQA dataset comprising 84K question-answer pairs that jointly encode global anatomical localization and local lesion characteristics, explicitly supporting multi-level diagnostic reasoning. We further propose region-zoom embeddings and a GRPO-based reinforcement learning framework to optimize fine-grained vision–language alignment without manual pixel-level annotations. Contribution/Results: Our approach achieves state-of-the-art performance on CT disease diagnosis tasks, significantly outperforming both general-purpose and existing medical VLMs. It demonstrates strong generalization across diverse anatomical regions and pathological conditions, validating the efficacy of coarse-to-fine reasoning and annotation-efficient alignment in medical VLMs.

Technology Category

• Computer Vision: Large Vision Models
• Knowledge Representation and Reasoning: Diagnosis and Abductive Reasoning
• Machine Learning: Calibration & Uncertainty Quantification

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets
• Web Mining and Content Analysis: Large pretrained models with web data

Abstract

General-purpose large Vision-Language Models (VLMs) demonstrate strong capabilities in generating detailed descriptions for natural images. However, their performance in the medical domain remains suboptimal, even for relatively straightforward tasks, primarily due to the lack of large-scale, high-quality, specialized medical imaging datasets and the neglect of the diagnostic process that progresses from coarse to fine-grained. To address the first issue, we construct the CT-RATE-VQA dataset, which has 84K QA pairs. For the second issue, we propose MedReason-R1, a medical VLM with explicit reasoning process for disease diagnosis. MedReason-R1 incorporates a novel strategy that embeds zoom-in disease region-of-interest areas into the image, highlighting the crucial role of both global localization and disease-specific details in enhancing the model's diagnostic performance. Furthermore, we introduce the GRPO reinforcement learning framework to MedReason-R1, which enables effective reasoning without relying on costly manual annotations. Compared to recent general-purpose and medical VLMs, MedReason-R1 achieves state-of-the-art performance in CT disease diagnosis while retaining generalization. The code, checkpoints, and dataset are available at: https://github.com/Leevan001/MedReason-R1

Problem

• Enhancing medical VLM diagnostic accuracy for CT images
• Addressing coarse-to-fine diagnostic process neglect in medical imaging
• Reducing reliance on costly manual annotations through reinforcement learning

Innovation

• Embedding zoom-in disease areas into CT images
• Using GRPO reinforcement learning for reasoning
• Constructing CT-RATE-VQA dataset with 84K QA pairs