Multimodal Routing for Interpretable, Robust, and Auditable Clinical Prediction

2026-07-10

AI Summary

Existing multimodal clinical prediction models often lack interpretability, robustness, and auditability when integrating electronic health records (EHR), making it difficult to quantify the contribution of individual modalities. This work proposes an explicit multimodal routing framework that models asymmetric interactions through unimodal, bimodal, and trimodal pathways and introduces a runtime routing mask mechanism to dynamically simulate modality absence and reweight pathways without retraining. Evaluated on MIMIC-IV, the approach combines structured longitudinal variables, clinical notes, and chest X-ray images to predict 25 phenotypes and ICU mortality. The method substantially enhances model transparency, robustness, and decision auditability, revealing distinct modality dependencies across diverse clinical scenarios.

Abstract

Electronic health record (EHR) data are inherently multimodal, and leveraging multiple modalities can improve predictive performance. However, most existing approaches rely on deep fusion, which obscures how individual modalities contribute to predictions and limits the interpretability of multimodal reasoning. We propose an explicit multimodal routing framework for clinical prediction that enables interpretable, robust, and auditable reasoning across three EHR modalities: structured longitudinal variables (L), clinical notes (N), and chest X-rays (I). Our model constructs discrete unimodal, directional bimodal, and trimodal routes to capture both individual modality signals and asymmetric cross-modal interactions. To audit multimodal reasoning and assess robustness, we introduce inference-time route masking, which simulates missing modalities and reweights the remaining routes without retraining. We analyze changes in performance and routing weights under these scenarios to understand model decision-making. We evaluate our framework on multi-label phenotype prediction (K = 25) and binary ICU mortality prediction using trimodal patient stays from MIMIC-IV, revealing systematic differences in modality reliance across clinical condition groups. Overall, our framework offers a transparent, auditable, and practical approach to multimodal clinical prediction, providing interpretability, robustness, and insights into how different data sources drive model decisions.

Problem

• multimodal fusion
• interpretability
• clinical prediction
• electronic health records
• model auditability

Innovation

• multimodal routing
• interpretable AI
• route masking
• clinical prediction
• cross-modal interaction