EyeMVP: OCT-Informed Fundus Representation Learning via Paired CFP--OCT Pretraining

2026-06-13

AI Summary

This study addresses the limited diagnostic performance of color fundus photography (CFP) in retinal disease screening due to its lack of depth-structured information. The authors propose the first large-scale cross-modal pretraining paradigm based on rigorously aligned, same-eye, same-day CFP–OCT triplets. By leveraging cross-modal masked reconstruction with a source-constrained cross-attention mechanism and CFP-derived structural masks, the method effectively mitigates inter-modal geometric misalignment and enables high-performance inference using only CFP inputs. Evaluated across 16 downstream tasks, the approach consistently outperforms existing models, achieving an AUROC of 0.948 for macular edema and 0.825 for myopic macular schisis—surpassing the performance of junior and mid-level ophthalmologists in several tasks.

Abstract

Color fundus photography (CFP) is the mainstay for large-scale retinal screening, yet its diagnostic capacity is constrained by the lack of depth-resolved structural information. Optical coherence tomography (OCT) provides cross-sectional retinal anatomy, but is less accessible in population-level screening. Here, we present EyeMVP, a cross-modal retinal foundation model that uses paired CFP--OCT pretraining to learn OCT-informed CFP representations. EyeMVP is pretrained on 674,893 strict same-eye same-day paired CFP--OCT image triples from 112,642 patients across eight hospitals in China. The model uses cross-modal masked reconstruction to enrich CFP representations with OCT-associated supervision, while requiring only CFP images at inference. To accommodate the non-aligned imaging geometry between en-face CFP and cross-sectional OCT, EyeMVP combines source-constrained cross-attention with CFP-derived structural masks. Across 16 downstream tasks, including classification, segmentation, few-shot adaptation, and cross-modal retrieval, EyeMVP outperforms representative retinal foundation models and shows consistent gains on tasks involving macular and optic nerve structure. For CFP-challenging macular diseases, EyeMVP achieves an AUROC of 0.948 for macular edema (vs.~0.852 for EyeCLIP) and 0.825 for myopic macular schisis. In an exploratory reader study, EyeMVP exceeds junior and intermediate ophthalmologist groups but does not reach senior ophthalmologist performance on macular edema, while showing numerically higher balanced accuracy than all reader groups on myopic macular schisis. These results suggest that pixel-level cross-modal reconstruction can enrich CFP representations with OCT-associated supervision, providing a practical route toward stronger CFP-based retinal analysis in screening settings.

Problem

• Color fundus photography
• Optical coherence tomography
• Retinal screening
• Cross-modal representation
• Macular diseases

Innovation

• cross-modal pretraining
• OCT-informed representation
• masked reconstruction
• retinal foundation model
• structural mask