Multi-modal Knowledge Decomposition based Online Distillation for Biomarker Prediction in Breast Cancer Histopathology

2025-08-24

AI Summary

To address the challenge of asynchronous acquisition of multimodal data (e.g., genomic profiles and histopathological images) in immunohistochemistry (IHC) biomarker prediction for breast cancer, this paper proposes a multimodal knowledge decomposition and collaborative online distillation framework. Methodologically, it introduces a modality-specific and shared feature disentanglement mechanism; designs a teacher–student architecture integrating similarity-preserving knowledge distillation (SKD) and collaborative online distillation (CLOD); and incorporates a multimodal loss optimization strategy enabling both unimodal and bimodal inference. Its key contribution lies in the first integration of knowledge decomposition with dynamic online distillation, substantially enhancing model generalizability under data scarcity. Evaluated on TCGA-BRCA and QHSU datasets, the method achieves state-of-the-art performance across critical IHC biomarker prediction tasks (ER, PR, HER2), setting new accuracy records for unimodal histopathological image-based prediction.

Technology Category

• Machine Learning: Multimodal Learning
• Computer Vision: Multi-modal Vision
• Data Mining & Knowledge Management: Mining of Visual, Multimedia & Multimodal Data

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs

Abstract

Immunohistochemical (IHC) biomarker prediction benefits from multi-modal data fusion analysis. However, the simultaneous acquisition of multi-modal data, such as genomic and pathological information, is often challenging due to cost or technical limitations. To address this challenge, we propose an online distillation approach based on Multi-modal Knowledge Decomposition (MKD) to enhance IHC biomarker prediction in haematoxylin and eosin (H&E) stained histopathology images. This method leverages paired genomic-pathology data during training while enabling inference using either pathology slides alone or both modalities. Two teacher and one student models are developed to extract modality-specific and modality-general features by minimizing the MKD loss. To maintain the internal structural relationships between samples, Similarity-preserving Knowledge Distillation (SKD) is applied. Additionally, Collaborative Learning for Online Distillation (CLOD) facilitates mutual learning between teacher and student models, encouraging diverse and complementary learning dynamics. Experiments on the TCGA-BRCA and in-house QHSU datasets demonstrate that our approach achieves superior performance in IHC biomarker prediction using uni-modal data. Our code is available at https://github.com/qiyuanzz/MICCAI2025_MKD.

Problem

• Enhancing IHC biomarker prediction in breast cancer histopathology
• Overcoming multi-modal data acquisition challenges in genomic-pathology fusion
• Enabling accurate biomarker inference using pathology-only data

Innovation

• Online distillation with multi-modal knowledge decomposition
• Similarity-preserving knowledge distillation maintains structural relationships
• Collaborative learning between teacher and student models