Language Agents for Hypothesis-driven Clinical Decision Making with Reinforcement Learning

2025-06-16

AI Summary

Existing LLM-based clinical decision-making methods either assume static, fully observed patient information or rely on zero-shot transfer, failing to capture physicians’ dynamic, iterative, and uncertainty-driven diagnostic reasoning. To address this, we propose LA-CDM—a hypothesis-driven, uncertainty-aware language agent that emulates clinicians’ closed-loop diagnostic process: actively querying patients, requesting critical tests, and progressively refining diagnoses. Methodologically, we introduce a novel tri-objective hybrid training paradigm jointly optimizing hypothesis generation, uncertainty estimation, and decision efficiency, integrating supervised learning with reinforcement learning for end-to-end optimization on the real-world MIMIC-CDM dataset. Evaluated on four abdominal disease diagnosis tasks, LA-CDM achieves significant improvements in both diagnostic accuracy and test-ordering efficiency. These results demonstrate the dual value of explicitly modeling clinical decision processes—enhancing both performance and interpretability.

Technology Category

• Reasoning under Uncertainty: Sequential Decision Making
• Knowledge Representation and Reasoning: Diagnosis and Abductive Reasoning
• Machine Learning: Calibration & Uncertainty Quantification

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information

Abstract

Clinical decision-making is a dynamic, interactive, and cyclic process where doctors have to repeatedly decide on which clinical action to perform and consider newly uncovered information for diagnosis and treatment. Large Language Models (LLMs) have the potential to support clinicians in this process, however, most applications of LLMs in clinical decision support suffer from one of two limitations: Either they assume the unrealistic scenario of immediate availability of all patient information and do not model the interactive and iterative investigation process, or they restrict themselves to the limited"out-of-the-box"capabilities of large pre-trained models without performing task-specific training. In contrast to this, we propose to model clinical decision-making for diagnosis with a hypothesis-driven uncertainty-aware language agent, LA-CDM, that converges towards a diagnosis via repeatedly requesting and interpreting relevant tests. Using a hybrid training paradigm combining supervised and reinforcement learning, we train LA-CDM with three objectives targeting critical aspects of clinical decision-making: accurate hypothesis generation, hypothesis uncertainty estimation, and efficient decision-making. We evaluate our methodology on MIMIC-CDM, a real-world dataset covering four abdominal diseases containing various clinical tests and show the benefit of explicitly training clinical decision-making for increasing diagnostic performance and efficiency.

Problem

• Model interactive clinical decision-making with hypothesis-driven agents
• Train language agents for accurate hypothesis and uncertainty estimation
• Improve diagnostic performance via reinforcement and supervised learning

Innovation

• Hypothesis-driven uncertainty-aware language agent
• Hybrid training with supervised and reinforcement learning
• Explicit training for clinical decision-making objectives