Rethinking Health Agents: From Siloed AI to Collaborative Decision Mediators

2026-03-25

AI Summary

This work addresses a critical limitation in current medical AI systems, which often operate in isolation and fragment situational awareness among patients, caregivers, and clinicians, thereby impeding collaborative alignment and shared goals. To bridge this gap, the paper introduces the first AI design framework explicitly oriented toward multi-stakeholder healthcare collaboration. The proposed approach reconceptualizes AI as a collaborative intermediary embedded within care interactions, integrating contextual information and reconciling disparate mental models to foster shared understanding—all while preserving human primacy in decision-making. Through a large language model–driven simulation grounded in clinically validated fictional cases of pediatric chronic kidney disease, the study demonstrates that adherence challenges stem from fragmented situational awareness and that conventional standalone AI tools are ill-equipped to address them. In contrast, the proposed framework significantly enhances multi-party alignment and collaborative efficacy.

Technology Category

• Humans and AI: Planning and Decision Support for Human-Machine Teams
• Multiagent Systems: Coordination and Collaboration
• Philosophy and Ethics of AI: Safety, Robustness & Trustworthiness

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• User Modeling, Personalization and Recommendation: Accountability, Transparency, and Ethics for personalization
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG

Abstract

Large language model based health agents are increasingly used by health consumers and clinicians to interpret health information and guide health decisions. However, most AI systems in healthcare operate in siloed configurations, supporting individual users rather than the multi-stakeholder relationships central to healthcare. Such use can fragment understanding and exacerbate misalignment among patients, caregivers, and clinicians. We reframe AI not as a standalone assistant, but as a collaborator embedded within multi-party care interactions. Through a clinically validated fictional pediatric chronic kidney disease case study, we show that breakdowns in adherence stem from fragmented situational awareness and misaligned goals, and that siloed use of general-purpose AI tools does little to address these collaboration gaps. We propose a conceptual framework for designing AI collaborators that surface contextual information, reconcile mental models, and scaffold shared understanding while preserving human decision authority.

Problem

• health agents
• collaborative decision-making
• siloed AI
• multi-stakeholder alignment
• shared understanding

Innovation

• collaborative AI
• health agents
• shared understanding
• multi-stakeholder alignment
• contextual mediation