Between Knowledge and Care: A Mixed-Methods Evaluation of Generative AI for T2DM Self-Management from Patient and Physician Perspectives

2026-07-04

AI Summary

This study evaluates the clinical applicability and clinician–patient acceptability of generative AI in delivering health guidance for self-management among individuals with type 2 diabetes. Employing a mixed-methods approach, the research first identifies patient information needs and then develops a five-dimensional physician evaluation framework grounded in accuracy, safety, clarity, completeness, and actionability to assess responses from four AI models through scoring and in-depth interviews. The work introduces two novel concepts—“pre-consultation scaffolding” and “fluency-induced hallucination”—to illuminate cognitive biases and contextual boundaries of AI in clinical settings. Findings indicate that AI performs well in factual explanations and lifestyle recommendations but exhibits notable deficiencies in medication-related reasoning and emotional support. Building on these insights, the study articulates four human-centered design principles to inform the responsible integration of generative AI into chronic disease management.

Abstract

Generative AI is increasingly used for everyday health guidance, yet its clinical appropriateness in chronic disease contexts remains poorly understood. This paper presents a two-part mixed-methods study on \revise{Type 2 Diabetes Mellitus (T2DM)}, examining how patients and physicians assess AI-generated health information. \revise{Study~1} analyzes 784 \revise{participant reported} patient queries to characterize seven informational need categories and \revise{develops a structured five dimensional physician rating rubric informed by patient query categories and clinician priorities} (\textit{Accuracy, Safety, Clarity, Integrity, Action Orientation}). \revise{Study~2} engages seven physicians scoring responses from four AI models and discussing evaluative reasoning through in-depth interviews. Models perform well on factual explanation and lifestyle guidance but consistently underperform on medication reasoning and emotional support. Two \revise{analytic concepts} emerge \revise{from the data}. The \textit{pre-visit primer} \revise{frames AI as preparation for clinical encounters rather than as a replacement for physicians}. The \textit{fluency illusion} \revise{describes how polished language may convey epistemic authority that the clinical content does not support}. Patients and physicians converged on three shared limitations (role boundaries, emotional inadequacy, personalization gaps) while diverging in evaluative emphasis, \revise{which informed} four design directions, task-aware orchestration, risk-aware fallback, dynamic personalization, and emotionally attuned interaction.

Problem

• Generative AI
• Type 2 Diabetes Mellitus
• Self-Management
• Clinical Appropriateness
• Health Information

Innovation

• generative AI
• mixed-methods evaluation
• T2DM self-management
• fluency illusion
• pre-visit primer