Evidence-based multimodal fusion on structured EHRs and free-text notes for ICU outcome prediction

2025-01-08

AI Summary

Addressing the challenges of integrating structured electronic health records (EHRs) with unstructured clinical notes and insufficient uncertainty modeling in ICU prognosis prediction, this paper proposes a novel multimodal fusion framework grounded in Dempster–Shafer (D-S) evidence theory. To our knowledge, this is the first work to apply D-S theory to EHR multimodal fusion. The framework jointly encodes heterogeneous data using LSTM and Transformer architectures, employs an evidence allocation network to generate modality-specific belief masses, fuses evidence via D-S combination rules, and incorporates an uncertainty calibration module for credibility-weighted, uncertainty-quantified predictions. Evaluated on MIMIC-III, the model achieves 1.05% and 1.02% improvements in balanced accuracy (BACC) for mortality and prolonged length-of-stay (PLOS) prediction, respectively; F1 scores increase by 9.74% and 6.04%; and Brier scores decrease by 26.8% and 15.1%. Critically, it substantially reduces false positive rates, enhancing both predictive reliability and clinical interpretability.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Reasoning under Uncertainty: Other Foundations of Reasoning under Uncertainty

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

Objective: Accurate Intensive Care Unit (ICU) outcome prediction is critical for improving patient treatment quality and ICU resource allocation. Existing research mainly focuses on structured data and lacks effective frameworks to integrate clinical notes from heterogeneous electronic health records (EHRs). This study aims to explore a multimodal framework based on evidence theory that can effectively combine heterogeneous structured EHRs and free-text notes for accurate and reliable ICU outcome prediction. Materials and Methods: We proposed an evidence-based multimodal fusion framework to predict ICU outcomes, including mortality and prolonged length of stay (PLOS), by utilizing both structured EHR data and free-text notes from the MIMIC-III database. We compare the performance against baseline models that use only structured EHRs, free-text notes, or existing multimodal approaches. Results: The results demonstrate that the evidence-based multimodal fusion model achieved both accurate and reliable prediction. Specifically, it outperformed the best baseline by 1.05%/1.02% in BACC, 9.74%/6.04% in F1 score, 1.28%/0.9% in AUROC, and 6.21%/2.68% in AUPRC for predicting mortality and PLOS, respectively. Additionally, it improved the reliability of the predictions with a 26.8%/15.1% reduction in the Brier score and a 25.0%/13.3% reduction in negative log-likelihood. Conclusion: This study demonstrates that the evidence-based multimodal fusion framework can serve as a strong baseline for predictions using structured EHRs and free-text notes. It effectively reduces false positives, which can help improve the allocation of medical resources in the ICU. This framework can be further applied to analyze multimodal EHRs for other clinical tasks.

Problem

• Health Information Integration
• ICU Patient Outcome Prediction
• Survival Status and Length of Stay

Innovation

• Evidence Theory Integration
• ICU Outcome Prediction
• Structured and Unstructured Data Fusion