Removing Temporal Note Redundancy Improves Multimodal Reinforcement Learning for Medicine

2026-08-14

AI Summary

This study addresses the issue of temporal redundancy in clinical notes undermining reinforcement learning (RL) state representations by proposing a redundancy-aware multimodal state framework. The method explicitly disentangles redundant information through singular value decomposition in embedding space and sentence-level differencing, thereby precisely extracting incremental clinical features to optimize mechanical ventilation decision-making. Evaluated on real-world ICU data using multiple off-policy evaluation metrics, the proposed framework significantly outperforms both structured and raw note baselines, effectively enhancing RL policy performance. By establishing a novel paradigm for handling temporal text redundancy, this work improves the robustness and practical utility of clinical decision support systems.

Abstract

Mechanical ventilation is a critical life-support intervention, requiring dynamic adjustments to ventilator settings as a patient's condition evolves. While reinforcement learning (RL) offers a promising framework for optimizing these sequential decisions, standard approaches rely primarily on structured electronic health record (EHR) data, missing crucial clinical context recorded in free-text notes. Integrating longitudinal clinical notes into RL state spaces is challenging because notes are heavily inflated by temporal redundancy, such as copy-forward text, templating, and repetitive documentation, which dilutes time-local updates and degrades state representation quality. To address this, we propose a redundancy-aware multimodal state representation framework that explicitly removes duplicated note text over time before policy learning. We evaluate two computationally efficient temporal decomposition strategies for removing duplicated note text: (1) an embedding-space decomposition using singular value decomposition on local history subspaces, and (2) an interpretable sentence-level diff operation that filters out previously documented sentences before text encoding. Using real-world ICU data, we demonstrate that state representations constructed by stripping temporal note redundancy significantly outperform both structured-only and raw-note baselines across multiple off-policy evaluation methods (Model-Based Rollouts, Fitted Q-Evaluation, Weighted Importance Sampling, and Weighted Doubly Robust Evaluation). Our findings show that explicitly isolating new clinical information from repeated note text yields higher-quality state representations and directly improves RL performance for clinical decision support.

Problem

• Multimodal Reinforcement Learning
• Temporal Note Redundancy
• State Representation
• Clinical Notes
• Mechanical Ventilation

Innovation

• Multimodal Reinforcement Learning
• Temporal Redundancy Removal
• Clinical Notes
• State Representation
• Mechanical Ventilation