Embracing Aleatoric Uncertainty in Medical Multimodal Learning with Missing Modalities

2026-01-29

AI Summary

This work addresses the challenge of missing modalities in medical multimodal learning, a problem exacerbated by the inherent uncertainty in clinical data acquisition that existing methods often overlook. To tackle this, the study proposes the first approach that explicitly models aleatoric uncertainty by representing each modality as a multivariate Gaussian distribution. It further introduces an uncertainty-aware dynamic message-passing mechanism over a patient–modality bipartite graph, enabling adaptive handling of missing modalities and reliable, uncertainty-informed fusion of available information. Evaluated on the MIMIC-IV and eICU datasets, the method achieves significant performance gains, improving AUC-ROC by 2.26% and 2.17%, respectively, thereby outperforming current state-of-the-art approaches.

Technology Category

• Machine Learning: Multimodal Learning
• Reasoning under Uncertainty: Uncertainty Representations
• Computer Vision: Multi-modal Vision

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Graph embeddings and representation learning for Web-related graphs
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG

Abstract

Medical multimodal learning faces significant challenges with missing modalities prevalent in clinical practice. Existing approaches assume equal contribution of modality and random missing patterns, neglecting inherent uncertainty in medical data acquisition. In this regard, we propose the Aleatoric Uncertainty Modeling (AUM) that explicitly quantifies unimodal aleatoric uncertainty to address missing modalities. Specifically, AUM models each unimodal representation as a multivariate Gaussian distribution to capture aleatoric uncertainty and enable principled modality reliability quantification. To adaptively aggregate captured information, we develop a dynamic message-passing mechanism within a bipartite patient-modality graph using uncertainty-aware aggregation mechanism. Through this process, missing modalities are naturally accommodated, while more reliable information from available modalities is dynamically emphasized to guide representation generation. Our AUM framework achieves an improvement of 2.26% AUC-ROC on MIMIC-IV mortality prediction and 2.17% gain on eICU, outperforming existing state-of-the-art approaches.

Problem

• missing modalities
• aleatoric uncertainty
• medical multimodal learning
• modality reliability
• uncertainty quantification

Innovation

• Aleatoric Uncertainty
• Multimodal Learning
• Missing Modalities
• Uncertainty-Aware Aggregation
• Gaussian Representation