When AI Writes Back: Ethical Considerations by Physicians on AI-Drafted Patient Message Replies

2025-08-17

AI Summary

This study addresses core ethical challenges associated with deploying generative AI (GenAI) for automated clinical patient messaging. Using qualitative methods, we conducted semi-structured interviews with 21 physicians participating in a GenAI pilot program, systematically identifying four critical ethical dimensions: the necessity of human oversight, transparency and informed consent regarding AI use, accurate perception of AI’s role, and robust privacy protection and data security. Contrary to prevailing assumptions, our findings establish that ethical accountability resides with AI users—not the technology itself—thereby challenging anthropomorphic attributions of responsibility. Based on this insight, we propose a practical, human–AI collaborative framework grounded in shared responsibility and clinical accountability. The study contributes actionable, ethics-informed guidance for the safe, trustworthy, and responsible integration of GenAI into clinical communication workflows.

Technology Category

• Philosophy and Ethics of AI: Safety, Robustness & Trustworthiness
• Natural Language Processing: Ethics — Bias, Fairness, Transparency & Privacy
• Machine Learning: Ethics, Bias, and Fairness

Application Category

• User Modeling, Personalization and Recommendation: Accountability, Transparency, and Ethics for personalization
• Economics, Online Markets and Human Computation: Trust and reliance of crowd workers and data experts on GenAI
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

The increasing burden of responding to large volumes of patient messages has become a key factor contributing to physician burnout. Generative AI (GenAI) shows great promise to alleviate this burden by automatically drafting patient message replies. The ethical implications of this use have however not been fully explored. To address this knowledge gap, we conducted a semi-structured interview study with 21 physicians who participated in a GenAI pilot program. We found that notable ethical considerations expressed by the physician participants included human oversight as ethical safeguard, transparency and patient consent of AI use, patient misunderstanding of AI's role, and patient privacy and data security as prerequisites. Additionally, our findings suggest that the physicians believe the ethical responsibility of using GenAI in this context primarily lies with users, not with the technology. These findings may provide useful insights into guiding the future implementation of GenAI in clinical practice.

Problem

• Investigating ethical implications of AI-drafted patient message replies
• Exploring physician perspectives on AI use in clinical communication
• Addressing oversight, transparency and privacy concerns in AI implementation

Innovation

• Semi-structured interviews with physicians
• Human oversight as ethical safeguard
• Transparency and patient consent requirements