Robustness and Scalability Of Machine Learning for Imbalanced Clinical Data in Emergency and Critical Care

2025-12-25

AI Summary

Clinical tabular data in emergency and critical care settings exhibit severe class imbalance, undermining the robustness and clinical utility of predictive models. This work systematically evaluates XGBoost, TabNet, and our newly proposed lightweight TabResNet across three dimensions: predictive accuracy, robustness to class imbalance (measured via imbalance ratio, geometric mean, and Brier score calibration), and computational scalability. We conduct the first unified quantification of imbalance and cross-task comparison across seven key clinical prediction tasks on MIMIC-IV-ED and eICU. TabResNet replaces TabNet’s computationally expensive attention mechanism with a residual architecture, significantly reducing inference cost; Bayesian hyperparameter optimization ensures fair model comparison. Results show that XGBoost achieves the highest stability across imbalance levels and best scalability; TabResNet matches TabNet’s imbalance robustness while offering superior efficiency. The findings affirm that architectural simplicity outperforms complexity, positioning tree-based models as the preferred choice for time-sensitive clinical applications.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Humans and AI: Explainable AI (XAI) for Human Understanding
• Data Mining & Knowledge Management: Scalability, Parallel & Distributed Systems

Application Category

• Web Mining and Content Analysis: Large pretrained models with web data
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Search and Retrieval-Augmented AI: Efficiency and scalability of Web search engines

Abstract

Emergency and intensive care environments require predictive models that are both accurate and computationally efficient, yet clinical data in these settings are often severely imbalanced. Such skewness undermines model reliability, particularly for rare but clinically crucial outcomes, making robustness and scalability essential for real-world usage. In this paper, we systematically evaluate the robustness and scalability of classical machine learning models on imbalanced tabular data from MIMIC-IV-ED and eICU. Class imbalance was quantified using complementary metrics, and we compared the performance of tree-based methods, the state-of-the-art TabNet deep learning model, and a custom lightweight residual network. TabResNet was designed as a computationally efficient alternative to TabNet, replacing its complex attention mechanisms with a streamlined residual architecture to maintain representational capacity for real-time clinical use. All models were optimized via a Bayesian hyperparameter search and assessed on predictive performance, robustness to increasing imbalance, and computational scalability. Our results, on seven clinically vital predictive tasks, show that tree-based methods, particularly XGBoost, consistently achieved the most stable performance across imbalance levels and scaled efficiently with sample size. Deep tabular models degraded more sharply under imbalance and incurred higher computational costs, while TabResNet provided a lighter alternative to TabNet but did not surpass ensemble benchmarks. These findings indicate that in emergency and critical care, robustness to imbalance and computational scalability could outweigh architectural complexity. Tree-based ensemble methods currently offer the most practical and clinically feasible choice, equipping practitioners with a framework for selecting models suited to high-stakes, time-sensitive environments.

Problem

• Evaluating robustness and scalability of ML models on imbalanced clinical data
• Comparing tree-based, deep learning, and custom models for emergency care predictions
• Identifying practical models for high-stakes, time-sensitive clinical environments

Innovation

• Evaluated tree-based models, TabNet, and lightweight TabResNet
• Designed TabResNet with residual architecture for computational efficiency
• Found tree-based methods most robust and scalable for imbalanced data