Multimodal Feature Prototype Learning for Interpretable and Discriminative Cancer Survival Prediction

2025-10-07

AI Summary

Current cancer survival prediction models suffer from poor interpretability and frequently neglect alignment between global tumor context and genomic semantics in multimodal learning. To address these limitations, we propose FeatProto—a unified multimodal feature prototype framework that jointly encodes whole-slide pathological images (capturing both global and local morphological patterns) and genomic data. Our approach introduces three key innovations: (1) a phenotypic representation fusion mechanism to bridge histopathological and molecular phenotypes; (2) exponential moving average–based prototype updating (EMA ProtoUp) for stable cross-modal alignment; and (3) a hierarchical prototype matching strategy to accommodate intra-tumor heterogeneity and model population-level survival patterns. Evaluated on four public cancer datasets, FeatProto consistently outperforms state-of-the-art unimodal and multimodal baselines, achieving significant improvements in both concordance index (C-index) and interpretability—including prototype traceability and clinical semantic consistency.

Technology Category

• Machine Learning: Multimodal Learning
• Computer Vision: Multi-modal Vision
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization

Abstract

Survival analysis plays a vital role in making clinical decisions. However, the models currently in use are often difficult to interpret, which reduces their usefulness in clinical settings. Prototype learning presents a potential solution, yet traditional methods focus on local similarities and static matching, neglecting the broader tumor context and lacking strong semantic alignment with genomic data. To overcome these issues, we introduce an innovative prototype-based multimodal framework, FeatProto, aimed at enhancing cancer survival prediction by addressing significant limitations in current prototype learning methodologies within pathology. Our framework establishes a unified feature prototype space that integrates both global and local features of whole slide images (WSI) with genomic profiles. This integration facilitates traceable and interpretable decision-making processes. Our approach includes three main innovations: (1) A robust phenotype representation that merges critical patches with global context, harmonized with genomic data to minimize local bias. (2) An Exponential Prototype Update Strategy (EMA ProtoUp) that sustains stable cross-modal associations and employs a wandering mechanism to adapt prototypes flexibly to tumor heterogeneity. (3) A hierarchical prototype matching scheme designed to capture global centrality, local typicality, and cohort-level trends, thereby refining prototype inference. Comprehensive evaluations on four publicly available cancer datasets indicate that our method surpasses current leading unimodal and multimodal survival prediction techniques in both accuracy and interoperability, providing a new perspective on prototype learning for critical medical applications. Our source code is available at https://github.com/JSLiam94/FeatProto.

Problem

• Enhancing cancer survival prediction using interpretable multimodal prototype learning
• Addressing limitations in prototype methods by integrating pathology and genomics
• Improving model accuracy and clinical interpretability for tumor heterogeneity analysis

Innovation

• Integrates global and local image features with genomic profiles
• Uses exponential prototype update strategy for tumor heterogeneity
• Implements hierarchical prototype matching for improved inference