ConceptM$^3$oE: Concept-Guided Multimodal Mixture of Experts for Interpretable Computational Pathology

2026-05-23

AI Summary

Current computational pathology models struggle to integrate multimodal diagnostic information and provide interpretable justifications for their decisions, particularly when morphological features alone are insufficient to distinguish complex tumor subtypes. This work proposes a concept-guided multimodal mixture-of-experts (MoE) architecture that, for the first time, embeds structured diagnostic concepts into the MoE framework. By decomposing multimodal evidence through modality-specific, redundant, and synergistic experts—and preserving original information via residual connections—the model achieves performance on par with unconstrained baselines on pediatric brain tumor and glioma datasets. Notably, it demonstrates over a 10% improvement in macro-F1 under few-shot settings, faster convergence, and generates reasoning trajectories validated by neuropathologists as clinically interpretable.

Abstract

Healthcare models are transitioning from unimodal prediction toward multimodal reasoning over heterogeneous diagnostic inputs. In computational pathology, for complex tumor subtypes where morphology alone can be challenging to distinguish, pathology reports and molecular measurements may provide additional diagnostic evidence alongside whole-slide images, yet existing models often fail to clarify how diverse signals assemble into recognizable diagnostic concepts. We propose ConceptM$^3$oE (Concept Multimodal MoE), which embeds concept formation directly within interaction-aware mixture-of-experts (MoE) pathways. The architecture decomposes evidence into modality-specific, redundant, and synergistic experts, which are then projected into structured concept bottlenecks mapping latent features to a hierarchy of morphology and biomarker concepts. To prevent the information loss typical of interpretable bottlenecks, we utilize residual pathways within each expert to allow task-relevant signals to flow both through the concepts and directly to the final task prediction, so that high performance is maintained alongside interpretability. Across an institutional pediatric brain tumor cohort and a public glioma cohort, the framework delivers competitive performance to unconstrained models while producing reasoning traces validated by an independent neuropathologist. In data-limited regimes, ConceptM$^3$oE improves limited-data performance, increasing macro-F1 from 56.41% to 66.70% at small training sizes compared to non-concept-informed baselines, while also showing faster training convergence consistent with the regularizing effect of concept learning. This work offers a scalable path toward high-performance medical AI that is inherently verifiable and better aligned with the complex decision-making of clinical practice.

Problem

• computational pathology
• multimodal reasoning
• interpretable AI
• diagnostic concepts
• tumor subtypes

Innovation

• Concept Bottleneck
• Multimodal Mixture of Experts
• Interpretable AI
• Computational Pathology
• Residual Concept Pathways