Learning optimal treatment strategies for intraoperative hypotension using deep reinforcement learning

2025-05-27

AI Summary

Intraoperative hypotension management remains largely empirical, increasing the risk of acute kidney injury (AKI). To address this, we propose a personalized, deep reinforcement learning–based drug dosing recommendation system. Leveraging over 50,000 real-world surgical episodes, the model integrates 16-dimensional temporal physiological and pharmacological features and—uniquely for large-scale clinical closed-loop dose optimization—employs an offline deep Q-network (DQN). It dynamically recommends intravenous fluid and vasopressor dosages to prevent intraoperative hypotension and postoperative AKI. The policy is both interpretable and quantitatively evaluable: on the test set, it achieves a 69% match rate with clinician vasopressor decisions; when its recommended doses outperform actual clinical regimens, policy value significantly improves; and it yields the lowest observed AKI incidence among evaluated strategies.

Technology Category

• Machine Learning: Deep Neural Architectures and Foundation Models
• Search and Optimization: Learning to Search
• Multiagent Systems: Adversarial Agents

Application Category

• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

Traditional methods of surgical decision making heavily rely on human experience and prompt actions, which are variable. A data-driven system generating treatment recommendations based on patient states can be a substantial asset in perioperative decision-making, as in cases of intraoperative hypotension, for which suboptimal management is associated with acute kidney injury (AKI), a common and morbid postoperative complication. We developed a Reinforcement Learning (RL) model to recommend optimum dose of intravenous (IV) fluid and vasopressors during surgery to avoid intraoperative hypotension and postoperative AKI. We retrospectively analyzed 50,021 surgeries from 42,547 adult patients who underwent major surgery at a quaternary care hospital between June 2014 and September 2020. Of these, 34,186 surgeries were used for model training and 15,835 surgeries were reserved for testing. We developed a Deep Q-Networks based RL model using 16 variables including intraoperative physiologic time series, total dose of IV fluid and vasopressors extracted for every 15-minute epoch. The model replicated 69% of physician's decisions for the dosage of vasopressors and proposed higher or lower dosage of vasopressors than received in 10% and 21% of the treatments, respectively. In terms of IV fluids, the model's recommendations were within 0.05 ml/kg/15 min of the actual dose in 41% of the cases, with higher or lower doses recommended for 27% and 32% of the treatments, respectively. The model resulted in a higher estimated policy value compared to the physicians' actual treatments, as well as random and zero-drug policies. AKI prevalence was the lowest in patients receiving medication dosages that aligned with model's decisions. Our findings suggest that implementation of the model's policy has the potential to reduce postoperative AKI and improve other outcomes driven by intraoperative hypotension.

Problem

• Optimizing IV fluid and vasopressor doses to prevent intraoperative hypotension
• Reducing postoperative acute kidney injury through data-driven treatment recommendations
• Improving surgical outcomes by replacing variable human decisions with RL models

Innovation

• Deep Q-Networks for optimal treatment recommendations
• Reinforcement Learning to manage intraoperative hypotension
• Data-driven model reduces postoperative acute kidney injury