Temporal Dynamic Embedding for Irregularly Sampled Time Series

2025-04-08

AI Summary

Irregular sampling in clinical time-series data introduces sparsity and dynamic dimensionality, posing challenges for conventional deep learning models. To address this, we propose Temporal Dynamic Embedding (TDE), a novel framework that models each clinical variable as a time-evolving embedding vector. Crucially, TDE enables stepwise aggregation over only the currently observed subset of variables—achieving, for the first time, neural network input dimensionality that dynamically varies with time. Unlike prior approaches, TDE avoids explicit imputation and fixed-dimensional representations; instead, it employs dynamic embedding updates and observation-aware adaptive aggregation for end-to-end modeling. Evaluated on PhysioNet 2012, MIMIC-III, and PhysioNet 2019, TDE matches or surpasses state-of-the-art imputation-based methods and SOTA models in predictive performance, while significantly improving training efficiency.

Technology Category

• Machine Learning: Time-Series/Data Streams
• Data Mining & Knowledge Management: Mining of Spatial, Temporal or Spatio-Temporal Data
• Cognitive Modeling & Cognitive Systems: Neural Spike Coding

Application Category

• Graph Algorithms and Modeling for the Web: Graph embeddings and representation learning for Web-related graphs
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

In several practical applications, particularly healthcare, clinical data of each patient is individually recorded in a database at irregular intervals as required. This causes a sparse and irregularly sampled time series, which makes it difficult to handle as a structured representation of the prerequisites of neural network models. We therefore propose temporal dynamic embedding (TDE), which enables neural network models to receive data that change the number of variables over time. TDE regards each time series variable as an embedding vector evolving over time, instead of a conventional fixed structured representation, which causes a critical missing problem. For each time step, TDE allows for the selective adoption and aggregation of only observed variable subsets and represents the current status of patient based on current observations. The experiment was conducted on three clinical datasets: PhysioNet 2012, MIMIC-III, and PhysioNet 2019. The TDE model performed competitively or better than the imputation-based baseline and several recent state-of-the-art methods with reduced training runtime.

Problem

• Handling sparse irregularly sampled time series data
• Enabling neural networks to process variable-changing data
• Eliminating missing data issues in clinical time series

Innovation

• Temporal dynamic embedding for irregular time series
• Selective adoption of observed variable subsets
• Competitive performance with reduced runtime