Explaining Digital Pathology Models via Clustering Activations

2025-11-18

AI Summary

In digital pathology, the limited interpretability of CNNs hinders their clinical adoption. To address this, we propose a global interpretability method based on unsupervised clustering of convolutional activation features. Unlike conventional saliency maps—which provide only local, pixel-level explanations—our approach structurally groups deep-layer activation patterns to visualize and semantically decode the model’s holistic decision logic. Evaluated on prostate cancer detection, the method identifies discriminative activation clusters strongly associated with clinically relevant histopathological patterns (e.g., glandular architectural abnormalities), thereby enhancing decision transparency and clinical trustworthiness. This work introduces, for the first time, systematic activation clustering as a principled framework for interpreting digital pathology CNNs, establishing a novel paradigm for the trustworthy deployment of high-stakes medical AI systems.

Technology Category

• Computer Vision: Interpretability, Explainability, and Transparency
• Machine Learning: Transparent, Interpretable, Explainable ML
• Natural Language Processing: Interpretability, Analysis, and Evaluation of NLP Models

Application Category

• Web Mining and Content Analysis: Normalization, clustering, classification, and summarization of Web text
• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization

Abstract

We present a clustering-based explainability technique for digital pathology models based on convolutional neural networks. Unlike commonly used methods based on saliency maps, such as occlusion, GradCAM, or relevance propagation, which highlight regions that contribute the most to the prediction for a single slide, our method shows the global behaviour of the model under consideration, while also providing more fine-grained information. The result clusters can be visualised not only to understand the model, but also to increase confidence in its operation, leading to faster adoption in clinical practice. We also evaluate the performance of our technique on an existing model for detecting prostate cancer, demonstrating its usefulness.

Problem

• Developing clustering-based explainability for digital pathology neural networks
• Providing global model behavior insights beyond single-slide saliency methods
• Enhancing clinical adoption confidence through interpretable prostate cancer detection clusters

Innovation

• Clustering activations to explain digital pathology models
• Revealing global model behavior beyond single-slide saliency
• Visualizing clusters to boost clinical adoption confidence