Complementary Human-AI Clinical Reasoning in Ophthalmology

2025-10-25

AI Summary

Global shortages of ophthalmology specialists severely impede access to vision impairment diagnosis and management. To address this, we propose a human–AI complementary diagnostic paradigm and introduce AMIE—a Gemini-based, medically fine-tuned dialogue model that integrates web retrieval, self-critical reasoning, and structured narrative generation to augment clinicians’ diagnostic reasoning. AMIE does not replace clinical judgment; instead, it re-ranks differential diagnoses, supplements recommended examinations and treatments, and thereby enhances decision consistency and plan completeness. Experiments demonstrate that AMIE achieves diagnostic accuracy comparable to physician baselines; when clinicians incorporate its outputs, correct diagnoses shift significantly higher in ranking, inter-physician diagnostic agreement improves, and robust complementary benefits persist across diverse clinical scenarios. This work constitutes the first empirical validation that large language model–driven structured reasoning support can substantively improve ophthalmologic clinical decision quality.

Technology Category

• Knowledge Representation and Reasoning: Diagnosis and Abductive Reasoning
• Humans and AI: AI for Accessibility
• Computer Vision: Diffusion Models for Vision

Application Category

• Economics, Online Markets and Human Computation: Trust and reliance of crowd workers and data experts on GenAI
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Search and Retrieval-Augmented AI: Agentic search

Abstract

Vision impairment and blindness are a major global health challenge where gaps in the ophthalmology workforce limit access to specialist care. We evaluate AMIE, a medically fine-tuned conversational system based on Gemini with integrated web search and self-critique reasoning, using real-world clinical vignettes that reflect scenarios a general ophthalmologist would be expected to manage. We conducted two complementary evaluations: (1) a human-AI interactive diagnostic reasoning study in which ophthalmologists recorded initial differentials and plans, then reviewed AMIE's structured output and revised their answers; and (2) a masked preference and quality study comparing AMIE's narrative outputs with case author reference answers using a predefined rubric. AMIE showed standalone diagnostic performance comparable to clinicians at baseline. Crucially, after reviewing AMIE's responses, ophthalmologists tended to rank the correct diagnosis higher, reached greater agreement with one another, and enriched their investigation and management plans. Improvements were observed even when AMIE's top choice differed from or underperformed the clinician baseline, consistent with a complementary effect in which structured reasoning support helps clinicians re-rank rather than simply accept the model output. Preferences varied by clinical grade, suggesting opportunities to personalise responses by experience. Without ophthalmology-specific fine-tuning, AMIE matched clinician baseline and augmented clinical reasoning at the point of need, motivating multi-axis evaluation, domain adaptation, and prospective multimodal studies in real-world settings.

Problem

• Addressing global ophthalmology workforce gaps limiting specialist care access
• Evaluating AI diagnostic performance compared to human ophthalmologists using clinical vignettes
• Investigating complementary human-AI reasoning to enhance clinical decision-making accuracy

Innovation

• Medically fine-tuned conversational AI system
• Integrated web search and self-critique reasoning
• Complementary human-AI interactive diagnostic evaluation