Reinforcement Learning over Patient Trajectories for Clinical Reasoning in EHR Foundation Models

2026-09-10

AI Summary

为提高EHR模型的临床推理能力，提出了一种基于强化学习的微调框架，通过设计时间敏感的奖励机制来优化患者轨迹预测任务。

Abstract

Electronic health record (EHR) foundation models trained on longitudinal patient trajectories have demonstrated strong performance across diverse clinical prediction tasks. However, their clinical reasoning capabilities remain constrained by next-token prediction on limited and incomplete EHR data. To address this, we propose a reinforcement learning (RL) fine-tuning framework that treats EHR foundation models as generative policies over patient trajectories. We formulate common clinical prediction problems (e.g., hospital readmission) as event-conditioned, time-windowed reasoning tasks. We then design time-aware, rollout-sensitive rewards to account for finite rollout lengths and temporally inconclusive outcomes. We find that RL fine-tuning consistently improves over pre-trained backbones and strong baselines. Notably, it enables smaller models to surpass larger pre-trained models in data-limited regimes and induces positive transfer across tasks. Further analysis shows that RL fine-tuned models generate trajectories with stronger structural and semantic alignment to ground truth and greater downstream utility.

Problem

• Reinforcement Learning
• Clinical Reasoning
• EHR Foundation Models
• Patient Trajectories

Innovation

• reinforcement learning
• EHR foundation models
• time-aware rewards
• clinical reasoning
• patient trajectories