A Survey of Multimodal Ophthalmic Diagnostics: From Task-Specific Approaches to Foundational Models

2025-07-31

AI Summary

Addressing the global challenge of insufficient diagnostic accuracy for visual impairment, this paper tackles the limitations of current ophthalmic AI systems in handling heterogeneous imaging modalities and clinical variability. Method: We present a systematic review of multimodal deep learning advances in ophthalmology up to 2025, advocating a paradigm shift from task-specific models to cross-modal foundation models. Our approach integrates self-supervised learning, attention-driven multimodal alignment, vision-language joint modeling, and large language model–augmented reasoning to jointly process fundus photography, optical coherence tomography (OCT), and angiography. We further introduce ultra-widefield imaging adaptation and reinforcement learning–guided decision support. Contribution: We establish the first comprehensive analytical framework covering datasets, evaluation protocols, and technical evolution; identify three critical bottlenecks—data inconsistency, weakly supervised annotation scarcity, and cross-device generalization—and outline future directions toward interpretable AI, automated structured reporting, and clinical closed-loop validation.

Technology Category

• Computer Vision: Multi-modal Vision
• Machine Learning: Multimodal Learning
• Humans and AI: Other Foundations of Human Computation & AI

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Visual impairment represents a major global health challenge, with multimodal imaging providing complementary information that is essential for accurate ophthalmic diagnosis. This comprehensive survey systematically reviews the latest advances in multimodal deep learning methods in ophthalmology up to the year 2025. The review focuses on two main categories: task-specific multimodal approaches and large-scale multimodal foundation models. Task-specific approaches are designed for particular clinical applications such as lesion detection, disease diagnosis, and image synthesis. These methods utilize a variety of imaging modalities including color fundus photography, optical coherence tomography, and angiography. On the other hand, foundation models combine sophisticated vision-language architectures and large language models pretrained on diverse ophthalmic datasets. These models enable robust cross-modal understanding, automated clinical report generation, and decision support. The survey critically examines important datasets, evaluation metrics, and methodological innovations including self-supervised learning, attention-based fusion, and contrastive alignment. It also discusses ongoing challenges such as variability in data, limited annotations, lack of interpretability, and issues with generalizability across different patient populations. Finally, the survey outlines promising future directions that emphasize the use of ultra-widefield imaging and reinforcement learning-based reasoning frameworks to create intelligent, interpretable, and clinically applicable AI systems for ophthalmology.

Problem

• Surveying multimodal deep learning in ophthalmology up to 2025
• Comparing task-specific methods and foundational models for diagnosis
• Addressing challenges like data variability and model interpretability

Innovation

• Task-specific multimodal deep learning approaches
• Large-scale multimodal foundational models
• Self-supervised learning and attention-based fusion