Agentifying Patient Dynamics within LLMs through Interacting with Clinical World Model

2026-05-14

AI Summary

This work addresses the challenge of modeling the causal relationships between dynamic physiological changes and clinical interventions in ICU sepsis management using large language models (LLMs). To this end, the authors propose SepsisAgent—the first framework integrating a clinical world model with an LLM-based agent. SepsisAgent generates treatment recommendations through a propose–simulate–optimize pipeline and employs a three-stage curriculum training strategy comprising supervised fine-tuning, behavioral cloning, and world-model-based reinforcement learning to enhance decision-making capabilities. Off-policy evaluation on the MIMIC-IV dataset demonstrates that SepsisAgent significantly outperforms existing reinforcement learning and LLM baselines across key safety metrics, including treatment value, adherence to clinical guidelines, and suppression of hazardous actions, while maintaining robust performance even when deployed without the simulator.

Abstract

Sepsis management in the ICU requires sequential treatment decisions under rapidly evolving patient physiology. Although large language models (LLMs) encode broad clinical knowledge and can reason over guidelines, they are not inherently grounded in action-conditioned patient dynamics. We introduce SepsisAgent, a world model-augmented LLM agent for sepsis treatment recommendation. SepsisAgent uses a learned Clinical World Model to simulate patient responses under candidate fluid--vasopressor interventions, and follows a propose--simulate--refine workflow before committing to a prescription. We first show that world-model access alone yields inconsistent LLM decision performance, motivating agent-specific training. We then train SepsisAgent through a three-stage curriculum: patient-dynamics supervised fine-tuning, propose--simulate--refine behavior cloning, and world-model-based agentic reinforcement learning. On MIMIC-IV sepsis trajectories, SepsisAgent outperforms all traditional RL and LLM-based baselines in off-policy value while achieving the best safety profile under guideline adherence and unsafe-action metrics. Further analysis shows that repeated interaction with the Clinical World Model enables the agent to learn regularities in patient evolution, which remain useful even when simulator access is removed.

Problem

• sepsis management
• patient dynamics
• clinical decision-making
• large language models
• treatment recommendation

Innovation

• Clinical World Model
• LLM Agent
• propose-simulate-refine
• agentic reinforcement learning
• sepsis management