A Domain Incremental Continual Learning Benchmark for ICU Time Series Model Transportability

2026-05-05

AI Summary

This study addresses the challenge that ICU time-series prediction models trained on large hospitals often fail to generalize to resource-constrained smaller facilities. To tackle this, the authors frame cross-institutional model transfer as a domain-incremental continual learning task, enabling adaptation to the target domain’s data distribution while preserving knowledge from the source domain. They establish the first benchmark for domain-incremental continual learning on ICU time-series data and systematically evaluate representative methods—such as experience replay and Elastic Weight Consolidation (EWC)—using real-world multicenter datasets to assess their capacity for knowledge retention and transfer. Their analysis reveals substantial differences in measurement distributions and frequencies across ICUs in distinct U.S. regions, thereby demonstrating both the promise and limitations of current continual learning approaches in clinical settings.

Abstract

In recent years, machine learning has made significant progress in clinical outcome prediction, demonstrating increasingly accurate results. However, the substantial resources required for hospitals to train these models, such as data collection, labeling, and computational power, limit the feasibility for smaller hospitals to develop their own models. An alternative approach involves transferring a machine learning model trained by a large hospital to smaller hospitals, allowing them to fine-tune the model on their specific patient data. However, these models are often trained and validated on data from a single hospital, raising concerns about their generalizability to new data. Our research shows that there are notable differences in measurement distributions and frequencies across various regions in the United States. To address this, we propose a benchmark that tests a machine learning model's ability to transfer from a source domain to different regions across the country. This benchmark assesses a model's capacity to learn meaningful information about each new domain while retaining key features from the original domain. Using this benchmark, we frame the transfer of a machine learning model from one region to another as a domain incremental learning problem. While the task of patient outcome prediction remains the same, the input data distribution varies, necessitating a model that can effectively manage these shifts. We evaluate two popular domain incremental learning methods: data replay, which stores examples from previous data sources for fine-tuning on the current source, and Elastic Weight Consolidation (EWC), a model parameter regularization method that maintains features important for both data sources.

Problem

• domain incremental learning
• model transportability
• ICU time series
• distribution shift
• continual learning

Innovation

• Domain Incremental Learning
• Continual Learning
• Model Transportability
• ICU Time Series
• Elastic Weight Consolidation