TAMER: A Test-Time Adaptive MoE-Driven Framework for EHR Representation Learning

2025-01-10

AI Summary

To address the degradation in generalization performance of electronic health record (EHR) modeling caused by patient population heterogeneity and distributional shift, this paper proposes TTA-MoE—the first test-time adaptive Mixture-of-Experts framework tailored for EHR sequence modeling. Unlike conventional methods, TTA-MoE dynamically adapts to the health state distribution of newly admitted patients during inference—without requiring ground-truth labels—enabling fine-grained subgroup representation learning and online model updating. It is architecture-agnostic, seamlessly integrating with backbone models such as RETAIN and GRU. Extensive experiments across four real-world EHR datasets demonstrate consistent improvements in mortality and readmission prediction, with average AUC gains of 1.8–3.2 percentage points. The implementation is publicly available.

Technology Category

• Machine Learning: Mixture of Experts (MoE)
• Humans and AI: Human-in-the-loop Machine Learning
• Cognitive Modeling & Cognitive Systems: Adaptive Behavior

Application Category

• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Economics, Online Markets and Human Computation: Trust and reliance of crowd workers and data experts on GenAI

Abstract

We propose TAMER, a Test-time Adaptive MoE-driven framework for EHR Representation learning. TAMER combines a Mixture-of-Experts (MoE) with Test-Time Adaptation (TTA) to address two critical challenges in EHR modeling: patient population heterogeneity and distribution shifts. The MoE component handles diverse patient subgroups, while TTA enables real-time adaptation to evolving health status distributions when new patient samples are introduced. Extensive experiments across four real-world EHR datasets demonstrate that TAMER consistently improves predictive performance for both mortality and readmission risk tasks when combined with diverse EHR modeling backbones. TAMER offers a promising approach for dynamic and personalized EHR-based predictions in practical clinical settings. Code is publicly available at https://github.com/yhzhu99/TAMER.

Problem

• Electronic Health Records
• Patient Health Information
• Temporal Health Status

Innovation

• TAMER
• Mixed Expertise Model
• Electronic Health Records Analysis