HDMoE: A Hierarchical Decoupling-Fusion Mixture-of-Experts Framework for Multimodal Cancer Survival Prediction

2026-05-20

AI Summary

This work addresses the challenge of effectively eliminating modality redundancy and modeling fine-grained intra- and inter-modal relationships in multimodal cancer survival prediction. To this end, the authors propose a hierarchical decoupling–fusion Mixture-of-Experts (MoE) framework: the first-level MoE removes intra-modal redundancy, while the second-level MoE, integrated with a Random Feature Recombination (RFR) module, enables fine-grained inter-modal decoupling and fusion. By designing shared and routed experts, the method enhances cross-modal interaction modeling while preserving discriminative information. Extensive experiments on a private liver cancer dataset and three public TCGA datasets demonstrate that the proposed approach significantly improves survival prediction accuracy.

Abstract

Multimodal survival prediction, a crucial yet challenging task, demands the integration of multimodal medical data (\eg Whole Slide Images (WSIs) and Genomic Profiles) to achieve accurate prognostic modeling. Given the inherent heterogeneity across modalities, the feature decoupling-fusion paradigm has emerged as a dominant approach. However, these methods have the following shortcomings: (1) fail to reduce the redundant information of modality features before decoupling, which negatively affects the feature decoupling and fusion effect;(2) lack the ability to model the fine-grained relationships of the features and capture the local information interactions between intra- and inter-modality features. To address these issues, we propose a \underline{H}ierarchical \underline{D}ecoupling-Fusion \underline{M}ixture-\underline{o}f-\underline{E}xperts (HDMoE) framework with two levels of MoE and \underline{R}andom \underline{F}eature \underline{R}eorganization (RFR) modules.In the first-level MoE, shared experts and routed experts are employed to remove redundant information and extract fine-grained specific features within each modality, while the second-level MoE facilitates fine-grained inter-modality feature decoupling. Besides, we design two RFR modules following each level of MoE to finely fuse intra- and inter-modality features, which can help the model capture more fine-grained relationships between modalities. Extensive experimental results on our private Liver Cancer (LC) and three TCGA public datasets confirm the effectiveness of our proposed method. Codes are available at https://github.com/ZJUMAI/HDMoE.

Problem

• Multimodal survival prediction
• Feature decoupling-fusion
• Redundant information
• Fine-grained relationships
• Inter-modality interaction

Innovation

• Mixture-of-Experts
• Feature Decoupling-Fusion
• Multimodal Learning
• Random Feature Reorganization
• Cancer Survival Prediction