FusionFM: Fusing Eye-specific Foundational Models for Optimized Ophthalmic Diagnosis

2025-08-14

AI Summary

Significant performance disparities, inconsistent cross-task generalization, and unclear ensemble potential exist among state-of-the-art ophthalmic foundation models. Method: We systematically evaluate and fuse four advanced models—RETFound, VisionFM, RetiZero, and DINORET—using two gated fusion strategies, benchmarking uniformly across multinational standardized datasets on glaucoma, diabetic retinopathy, age-related macular degeneration (AMD), diabetes, and hypertension prediction using AUC and F1 scores. Results: RetiZero achieves the strongest cross-dataset generalization, while DINORET and RetiZero jointly attain top overall performance. Ensemble models yield modest yet robust improvements in glaucoma, AMD, and hypertension prediction (+0.5–1.2% AUC), though systemic disease prediction—particularly externally sourced hypertension—remains challenging. This work establishes the first comprehensive evaluation and fusion framework for ophthalmic foundation models spanning both ocular and systemic diseases, providing empirical evidence and a methodological paradigm for clinically deployable multi-task diagnostic systems.

Technology Category

• Computer Vision: Other Foundations of Computer Vision
• Machine Learning: Other Foundations of Machine Learning
• Humans and AI: Other Foundations of Human Computation & AI

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Economics, Online Markets and Human Computation: Trust and reliance of crowd workers and data experts on GenAI

Abstract

Foundation models (FMs) have shown great promise in medical image analysis by improving generalization across diverse downstream tasks. In ophthalmology, several FMs have recently emerged, but there is still no clear answer to fundamental questions: Which FM performs the best? Are they equally good across different tasks? What if we combine all FMs together? To our knowledge, this is the first study to systematically evaluate both single and fused ophthalmic FMs. To address these questions, we propose FusionFM, a comprehensive evaluation suite, along with two fusion approaches to integrate different ophthalmic FMs. Our framework covers both ophthalmic disease detection (glaucoma, diabetic retinopathy, and age-related macular degeneration) and systemic disease prediction (diabetes and hypertension) based on retinal imaging. We benchmarked four state-of-the-art FMs (RETFound, VisionFM, RetiZero, and DINORET) using standardized datasets from multiple countries and evaluated their performance using AUC and F1 metrics. Our results show that DINORET and RetiZero achieve superior performance in both ophthalmic and systemic disease tasks, with RetiZero exhibiting stronger generalization on external datasets. Regarding fusion strategies, the Gating-based approach provides modest improvements in predicting glaucoma, AMD, and hypertension. Despite these advances, predicting systemic diseases, especially hypertension in external cohort remains challenging. These findings provide an evidence-based evaluation of ophthalmic FMs, highlight the benefits of model fusion, and point to strategies for enhancing their clinical applicability.

Problem

• Evaluating single and fused foundational models for ophthalmic diagnosis
• Comparing performance across eye diseases and systemic conditions prediction
• Developing fusion strategies to enhance clinical diagnostic accuracy

Innovation

• Fusing multiple eye-specific foundational models
• Using Gating-based approach for model fusion
• Evaluating both ophthalmic and systemic disease prediction