Towards Interpretable Foundation Models for Retinal Fundus Images

2026-03-19

AI Summary

This work addresses the critical lack of interpretability in existing medical imaging foundation models, which hinders their trustworthiness and validation in high-stakes clinical settings. To this end, the authors propose Dual-IFM, an interpretable foundation model tailored for color fundus photographs, which uniquely integrates dual interpretability mechanisms within a foundation model architecture: local explanations via class evidence maps and global representation visualization through a two-dimensional projection layer. Leveraging self-supervised learning, Dual-IFM is pretrained on over 800,000 unlabeled images and achieves performance comparable to state-of-the-art models while using only 1/16 of their parameter count. Furthermore, it delivers reliable and interpretable predictions on out-of-distribution data, enhancing its clinical applicability.

Technology Category

• Computer Vision: Interpretability, Explainability, and Transparency
• Machine Learning: Deep Neural Architectures and Foundation Models
• Natural Language Processing: Interpretability, Analysis, and Evaluation of NLP Models

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization
• Web Mining and Content Analysis: Large pretrained models with web data

Abstract

Foundation models are used to extract transferable representations from large amounts of unlabeled data, typically via self-supervised learning (SSL). However, many of these models rely on architectures that offer limited interpretability, which is a critical issue in high-stakes domains such as medical imaging. We propose Dual-IFM, a foundation model that is interpretable-by-design in two ways: First, it provides local interpretability for individual images through class evidence maps that are faithful to the decision-making process. Second, it provides global interpretability for entire datasets through a 2D projection layer that allows for direct visualization of the model's representation space. We trained our model on over 800,000 color fundus photography from various sources to learn generalizable, interpretable representations for different downstream tasks. Our results show that our model reaches a performance range similar to that of state-of-the-art foundation models with up to $16\times$ the number of parameters, while providing interpretable predictions on out-of-distribution data. Our results suggest that large-scale SSL pretraining paired with inherent interpretability can lead to robust representations for retinal imaging.

Problem

• interpretable foundation models
• retinal fundus images
• medical imaging
• self-supervised learning
• model interpretability

Innovation

• interpretable foundation model
• self-supervised learning
• retinal fundus imaging
• class evidence maps
• representation visualization