Coefficient of Variation Masking: A Volatility-Aware Strategy for EHR Foundation Models

2025-12-04

AI Summary

Existing EHR foundation models apply uniform random masking to laboratory measurements, ignoring their heterogeneous variability: stable biomarkers (e.g., sodium) exhibit low clinical volatility, whereas highly variable ones (e.g., lactate) often signal acute pathophysiological changes and thus demand finer-grained modeling. To address this, we propose the first coefficient-of-variation (CV)-aware masking strategy within a masked autoencoder framework, dynamically increasing the masking probability for high-CV lab values and adopting a value-only reconstruction objective. This is the first work to explicitly incorporate CV into pretraining masking mechanisms. Evaluated on large-scale real-world EHR data, our method achieves a 12.7% reduction in reconstruction error, an average 3.4 percentage-point improvement in downstream task AUC, and 21% faster convergence—outperforming both standard random masking and variance-based alternatives.

Technology Category

• Machine Learning: Deep Generative Models & Autoencoders
• Computer Vision: Learning & Optimization for CV
• Humans and AI: Human-in-the-loop Machine Learning

Application Category

• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Masked autoencoders (MAEs) are increasingly applied to electronic health records (EHR) for learning general-purpose representations that support diverse clinical tasks. However, existing approaches typically rely on uniform random masking, implicitly assuming all features are equally predictable. In reality, laboratory tests exhibit substantial heterogeneity in volatility: some biomarkers (e.g., sodium) remain stable, while others (e.g., lactate) fluctuate considerably and are more difficult to model. Clinically, volatile biomarkers often signal acute pathophysiology and require more sophisticated modeling to capture their complex temporal patterns. We propose a volatility-aware pretraining strategy, Coefficient of Variation Masking (CV-Masking), that adaptively adjusts masking probabilities according to the intrinsic variability of each feature. Combined with a value-only masking objective aligned with clinical workflows, CV-Masking yields systematic improvements over random and variance-based strategies. Experiments on a large panel of laboratory tests show that CV-Masking enhances reconstruction, improves downstream predictive performance, and accelerates convergence, producing more robust and clinically meaningful EHR representations.

Problem

• Develops volatility-aware masking for EHR foundation models
• Addresses heterogeneity in biomarker predictability in clinical data
• Improves representation learning for volatile and stable biomarkers

Innovation

• Adaptive masking based on feature volatility
• Value-only masking aligned with clinical workflows
• Improves reconstruction, prediction, and convergence