CX-Mind: A Pioneering Multimodal Large Language Model for Interleaved Reasoning in Chest X-ray via Curriculum-Guided Reinforcement Learning

2025-07-31

AI Summary

Current chest X-ray (CXR) multi-task diagnostic models rely on single-step generation, lacking verifiable reasoning traces—leading to challenges in long-horizon reasoning, sparse reward signals, and frequent hallucinations. To address these limitations, we propose the first multimodal large language model (MLLM) supporting interleaved “think-then-answer” reasoning for CXR diagnosis. Our method introduces a novel clinical-report-guided procedural supervision framework and curriculum-guided reinforcement learning (CuRL-VPR), optimized via Group Relative Policy Optimization in two stages—without requiring a pretrained reward model. We construct CX-Set, a large-scale instruction-tuning dataset comprising 700K CXR images. Extensive experiments demonstrate an average 25.1% improvement across visual understanding, text generation, and spatiotemporal alignment tasks. On the Rui-CXR dataset, our model achieves significantly higher recall across all 14 disease categories. Multi-center expert evaluations further validate its clinical utility and interpretability.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Computer Vision: Multi-modal Vision
• Knowledge Representation and Reasoning: Diagnosis and Abductive Reasoning

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

Chest X-ray (CXR) imaging is one of the most widely used diagnostic modalities in clinical practice, encompassing a broad spectrum of diagnostic tasks. Recent advancements have seen the extensive application of reasoning-based multimodal large language models (MLLMs) in medical imaging to enhance diagnostic efficiency and interpretability. However, existing multimodal models predominantly rely on "one-time" diagnostic approaches, lacking verifiable supervision of the reasoning process. This leads to challenges in multi-task CXR diagnosis, including lengthy reasoning, sparse rewards, and frequent hallucinations. To address these issues, we propose CX-Mind, the first generative model to achieve interleaved "think-answer" reasoning for CXR tasks, driven by curriculum-based reinforcement learning and verifiable process rewards (CuRL-VPR). Specifically, we constructed an instruction-tuning dataset, CX-Set, comprising 708,473 images and 2,619,148 samples, and generated 42,828 high-quality interleaved reasoning data points supervised by clinical reports. Optimization was conducted in two stages under the Group Relative Policy Optimization framework: initially stabilizing basic reasoning with closed-domain tasks, followed by transfer to open-domain diagnostics, incorporating rule-based conditional process rewards to bypass the need for pretrained reward models. Extensive experimental results demonstrate that CX-Mind significantly outperforms existing medical and general-domain MLLMs in visual understanding, text generation, and spatiotemporal alignment, achieving an average performance improvement of 25.1% over comparable CXR-specific models. On real-world clinical dataset (Rui-CXR), CX-Mind achieves a mean recall@1 across 14 diseases that substantially surpasses the second-best results, with multi-center expert evaluations further confirming its clinical utility across multiple dimensions.

Problem

• Lack of verifiable reasoning in CXR diagnosis models
• Challenges in multi-task CXR diagnosis efficiency
• Need for improved clinical utility in MLLMs

Innovation

• Curriculum-guided reinforcement learning for reasoning
• Verifiable process rewards for supervision
• Interleaved think-answer reasoning model