Enhancing LLMs' Clinical Reasoning with Real-World Data from a Nationwide Sepsis Registry

2025-05-05

AI Summary

Clinical large language models (LLMs) suffer from limited clinical reasoning capabilities due to insufficient training on real-world, high-fidelity clinical data. To address this, we propose C-Reason—a novel framework that constructs reasoning-intensive questions from the nationally representative, privacy-compliant, high-quality, and large-scale Sepsis Registry, and fine-tunes a lightweight Phi-4 model via Reinforcement Learning from Human Feedback (RLHF). This work marks the first demonstration of real-world sepsis data–driven clinical reasoning enhancement in LLMs, yielding statistically significant performance gains on domain-specific benchmarks (p < 0.01) and receiving strong endorsement from clinical experts. Moreover, C-Reason exhibits robust generalization across tasks (e.g., antibiotic consultation), patient populations, and disease conditions (multi-disease reasoning). Its core innovation lies in systematically integrating high-fidelity real-world clinical data into the LLM reasoning training loop—establishing a reproducible, scalable data–algorithm co-design paradigm for clinical AI.

Technology Category

• Cognitive Modeling & Cognitive Systems: Conceptual Inference and Reasoning
• Machine Learning: Large Multimodal Models (LMMs)
• Knowledge Representation and Reasoning: Common-Sense Reasoning

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Search and Retrieval-Augmented AI: Large language models for search
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

Although large language models (LLMs) have demonstrated impressive reasoning capabilities across general domains, their effectiveness in real-world clinical practice remains limited. This is likely due to their insufficient exposure to real-world clinical data during training, as such data is typically not included due to privacy concerns. To address this, we propose enhancing the clinical reasoning capabilities of LLMs by leveraging real-world clinical data. We constructed reasoning-intensive questions from a nationwide sepsis registry and fine-tuned Phi-4 on these questions using reinforcement learning, resulting in C-Reason. C-Reason exhibited strong clinical reasoning capabilities on the in-domain test set, as evidenced by both quantitative metrics and expert evaluations. Furthermore, its enhanced reasoning capabilities generalized to a sepsis dataset involving different tasks and patient cohorts, an open-ended consultations on antibiotics use task, and other diseases. Future research should focus on training LLMs with large-scale, multi-disease clinical datasets to develop more powerful, general-purpose clinical reasoning models.

Problem

• Enhancing LLMs' clinical reasoning with real-world sepsis data
• Addressing limited LLM effectiveness in clinical practice
• Improving generalizability of clinical reasoning across diseases

Innovation

• Leveraging nationwide sepsis registry data
• Fine-tuning Phi-4 with reinforcement learning
• Developing C-Reason for clinical reasoning