Triple-Phase Multimodal Knowledge Aggregation Framework for Microbial Keratitis Subtype Diagnosis on Slit-Lamp Photography

2026-07-04

AI Summary

This study addresses the challenge of rapidly differentiating bacterial from fungal keratitis by proposing a three-stage multimodal knowledge fusion framework that integrates blue-light, scleral scatter, white-light slit-lamp images, and clinical metadata. Leveraging cross-modal contrastive learning, modality-specific fine-tuning, and feature-level multimodal ensemble, the framework achieves patient-level diagnostic precision. The work introduces an innovative tri-phase fusion strategy and a resampling-based balanced evaluation protocol to uncover the impact of multicenter data distribution shifts on model generalization, thereby establishing a more reliable assessment paradigm. Evaluated on a large multicenter dataset comprising 1,645 patients and 17,158 images, the method achieves 85.84% accuracy, an average F1-score of 84.46%, and an AUC of 0.885, significantly outperforming existing approaches.

Abstract

Microbial keratitis requires rapid pathogen identification to guide treatment, but culture- and PCR-based diagnostics are slow and resource-intensive. We developed a triple-phase multimodal framework for bacterial-versus-fungal keratitis classification using slit-lamp photographs acquired under blue-light, sclerotic-scatter, and white-light illumination, together with clinical metadata. The model combines cross-modality contrastive learning, modality-specific fine-tuning, and feature-level multimodal ensemble learning for patient-level prediction. We evaluated the framework on a multicenter dataset of 1,645 patients and 17,158 images from India and the United States. The model achieved 85.84% accuracy, 84.46% average F1-score, and 0.885 AUC. Site-specific evaluation showed that pooled results were overly optimistic, whereas resampling- and balance-based re-evaluation provided a more realistic assessment of cross-site generalization. Under all settings, our framework remained the top-performing approach. Upon acceptance, the code will be released and dataset access will be provided subject to University of Michigan data-sharing clearance.

Problem

• Microbial keratitis
• Pathogen identification
• Bacterial versus fungal classification
• Slit-lamp photography
• Diagnostic delay

Innovation

• multimodal learning
• contrastive learning
• ensemble learning
• microbial keratitis
• slit-lamp photography