Label-free Concept Based Multiple Instance Learning for Gigapixel Histopathology

2025-01-06

AI Summary

To address the scarcity of expert annotations and limited interpretability in whole-slide image (WSI) classification, this paper proposes the first label-free, concept-driven multiple instance learning (MIL) framework. Methodologically, it leverages vision-language models (e.g., CLIP) to automatically discover semantically meaningful medical concepts; predictions are generated via top-K patch aggregation and linear concept combination, yielding inherently interpretable outputs—without requiring predefined pathological concepts or pixel-level annotations. Key contributions include: (1) the first integration of vision-language priors into WSI modeling, enabling pathology-level (rather than pixel-level) interpretability; and (2) naturally traceable predictions that quantify each concept’s contribution. On Camelyon16 and PANDA, the method achieves AUC and accuracy >0.9; 87.1% and 85.3% of top-20 patches localize within tumor regions. A user study confirms strong alignment between discovered concepts and pathologists’ domain knowledge.

Technology Category

• Machine Learning: Multi-instance/Multi-view Learning
• Computer Vision: Interpretability, Explainability, and Transparency
• Cognitive Modeling & Cognitive Systems: Conceptual Inference and Reasoning

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Web Mining and Content Analysis: Large pretrained models with web data
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Multiple Instance Learning (MIL) methods allow for gigapixel Whole-Slide Image (WSI) analysis with only slide-level annotations. Interpretability is crucial for safely deploying such algorithms in high-stakes medical domains. Traditional MIL methods offer explanations by highlighting salient regions. However, such spatial heatmaps provide limited insights for end users. To address this, we propose a novel inherently interpretable WSI-classification approach that uses human-understandable pathology concepts to generate explanations. Our proposed Concept MIL model leverages recent advances in vision-language models to directly predict pathology concepts based on image features. The model's predictions are obtained through a linear combination of the concepts identified on the top-K patches of a WSI, enabling inherent explanations by tracing each concept's influence on the prediction. In contrast to traditional concept-based interpretable models, our approach eliminates the need for costly human annotations by leveraging the vision-language model. We validate our method on two widely used pathology datasets: Camelyon16 and PANDA. On both datasets, Concept MIL achieves AUC and accuracy scores over 0.9, putting it on par with state-of-the-art models. We further find that 87.1% (Camelyon16) and 85.3% (PANDA) of the top 20 patches fall within the tumor region. A user study shows that the concepts identified by our model align with the concepts used by pathologists, making it a promising strategy for human-interpretable WSI classification.

Problem

• Medical Imaging
• Interpretable AI
• Unsupervised Learning

Innovation

• Self-Supervised Learning
• Medical Concept Extraction
• Transparent Decision Process