To Recommend or Not to Recommend: Designing and Evaluating AI-Enabled Decision Support for Time-Critical Medical Events

2025-05-17

AI Summary

This study addresses socio-technical barriers—limited information, accuracy–timeliness trade-offs, and clinical cognitive divergence—that hinder AI-assisted decision-making in trauma resuscitation. We designed and evaluated a human–AI collaborative decision support system tailored to acute-care settings. Through medical human factors engineering, multi-center qualitative user research (N=35), and an online controlled experiment, we identified clinicians’ information needs and optimal AI output formats, and comparatively assessed two interaction paradigms: “information synthesis” versus “information + recommendation.” Our work provides the first empirical evidence of the accuracy–timeliness trade-off mechanism for AI recommendations in time-critical care and reveals clinician-perceived polarization between physicians and nurses. We propose three emergency-oriented human–AI collaboration design principles. Results show that “information + recommendation” significantly improves correct decision rates and identifies two core barriers to AI adoption: interpretability deficits and role-based trust asymmetries.

Technology Category

• Humans and AI: Planning and Decision Support for Human-Machine Teams
• Philosophy and Ethics of AI: Safety, Robustness & Trustworthiness
• Cognitive Modeling & Cognitive Systems: Simulating Human Behavior

Application Category

• User Modeling, Personalization and Recommendation: Accountability, Transparency, and Ethics for personalization
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

AI-enabled decision-support systems aim to help medical providers rapidly make decisions with limited information during medical emergencies. A critical challenge in developing these systems is supporting providers in interpreting the system output to make optimal treatment decisions. In this study, we designed and evaluated an AI-enabled decision-support system to aid providers in treating patients with traumatic injuries. We first conducted user research with physicians to identify and design information types and AI outputs for a decision-support display. We then conducted an online experiment with 35 medical providers from six health systems to evaluate two human-AI interaction strategies: (1) AI information synthesis and (2) AI information and recommendations. We found that providers were more likely to make correct decisions when AI information and recommendations were provided compared to receiving no AI support. We also identified two socio-technical barriers to providing AI recommendations during time-critical medical events: (1) an accuracy-time trade-off in providing recommendations and (2) polarizing perceptions of recommendations between providers. We discuss three implications for developing AI-enabled decision support used in time-critical events, contributing to the limited research on human-AI interaction in this context.

Problem

• Designing AI support for time-critical medical decisions
• Evaluating AI recommendations vs. information synthesis
• Addressing socio-technical barriers in emergency AI systems

Innovation

• AI-enabled decision-support for medical emergencies
• User research to design AI outputs
• Evaluated AI information and recommendations strategies