MIRA: Medical Time Series Foundation Model for Real-World Health Data

2025-06-09

AI Summary

To address challenges in real-world clinical time series—including irregular sampling, multi-source heterogeneity, and high missingness—this paper introduces the first unified foundation model tailored for clinical data. Methodologically, it proposes three key innovations: (1) Continuous-Time Rotary Position Embedding (CT-RoPE) for precise temporal alignment; (2) Frequency-Specific Mixture-of-Experts (FS-MoE) layers to capture multiscale dynamics; and (3) a Neural ODE–based dynamical extrapolation module for continuous-time forecasting at arbitrary timestamps. The model is pretrained on 454 billion clinical timepoints, achieving state-of-the-art performance: average out-of-distribution and in-distribution prediction errors decrease by 10% and 7%, respectively, under zero-shot and fine-tuned settings. Additionally, we release a comprehensive clinical time-series benchmark covering diverse tasks, enabling robust cross-institutional, cross-modal, and zero-shot transfer evaluation. This work advances continuous-time representation learning and generalizable forecasting in healthcare AI.

Technology Category

• Machine Learning: Time-Series/Data Streams
• Cognitive Modeling & Cognitive Systems: Other Foundations of Cognitive Modeling & Systems
• Computer Vision: Diffusion Models for Vision

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Web Mining and Content Analysis: Large pretrained models with web data

Abstract

A unified foundation model for medical time series -- pretrained on open access and ethics board-approved medical corpora -- offers the potential to reduce annotation burdens, minimize model customization, and enable robust transfer across clinical institutions, modalities, and tasks, particularly in data-scarce or privacy-constrained environments. However, existing generalist time series foundation models struggle to handle medical time series data due to their inherent challenges, including irregular intervals, heterogeneous sampling rates, and frequent missing values. To address these challenges, we introduce MIRA, a unified foundation model specifically designed for medical time series forecasting. MIRA incorporates a Continuous-Time Rotary Positional Encoding that enables fine-grained modeling of variable time intervals, a frequency-specific mixture-of-experts layer that routes computation across latent frequency regimes to further promote temporal specialization, and a Continuous Dynamics Extrapolation Block based on Neural ODE that models the continuous trajectory of latent states, enabling accurate forecasting at arbitrary target timestamps. Pretrained on a large-scale and diverse medical corpus comprising over 454 billion time points collect from publicly available datasets, MIRA achieves reductions in forecasting errors by an average of 10% and 7% in out-of-distribution and in-distribution scenarios, respectively, when compared to other zero-shot and fine-tuned baselines. We also introduce a comprehensive benchmark spanning multiple downstream clinical tasks, establishing a foundation for future research in medical time series modeling.

Problem

• Handles irregular intervals and missing values in medical time series
• Reduces annotation burdens and enables robust transfer across institutions
• Improves forecasting accuracy in data-scarce or privacy-constrained environments

Innovation

• Continuous-Time Rotary Positional Encoding for irregular intervals
• Frequency-specific mixture-of-experts layer for temporal specialization
• Neural ODE-based Continuous Dynamics Extrapolation Block for forecasting