Quasi-multimodal-based pathophysiological feature learning for retinal disease diagnosis.

2026-01-01

Medical Image Analysis

AI Summary

This study addresses the challenges in retinal disease diagnosis posed by the heterogeneity, invasiveness, and registration difficulties of ophthalmic multimodal data. To circumvent the need for real multimodal acquisition, the authors propose a unified framework that constructs quasi-multimodal inputs by synthesizing fluorescein angiography (FFA), multispectral imaging (MSI), and saliency maps. A parallel deep network architecture learns modality-specific features, while a cross-modal adaptive calibration mechanism enables flexible fusion and information pruning, significantly enhancing lesion representation. Evaluated on two public datasets, the method achieves state-of-the-art performance: a multi-label classification F1-score of 0.683 (AUC 0.953) and a diabetic retinopathy grading accuracy of 0.842 (Cohen’s Kappa 0.861), demonstrating both high diagnostic accuracy and clinical practicality.

Technology Category

• Computer Vision: Multi-modal Vision
• Machine Learning: Multimodal Learning
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Security and Privacy: Data transparency and provenance
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Problem

• retinal disease diagnosis
• multimodal data
• data heterogeneity
• image registration
• ophthalmic imaging

Innovation

• quasi-multimodal learning
• multimodal synthesis
• adaptive feature calibration
• pathophysiological representation
• retinal disease diagnosis