Beyond Observations: Reconstruction Error-Guided Irregularly Sampled Time Series Representation Learning

2025-11-10

AI Summary

Existing methods for irregularly sampled time series (ISTS) modeling overlook reconstruction error—a critical learning signal—leading to suboptimal representation learning and poor generalization, especially in missing-data regions. Method: We propose iTimER, a self-supervised pretraining framework that explicitly models reconstruction error as a noise-aware learning objective. iTimER estimates the error distribution, generates pseudo-observations via error-aware mixup augmentation, and aligns real and pseudo error distributions using the Wasserstein distance. It further incorporates contrastive learning to enhance representation discriminability. Contribution/Results: Without requiring any task-specific labels, iTimER achieves state-of-the-art performance across three fundamental tasks—classification, imputation, and forecasting—demonstrating superior modeling capability for missing regions and strong cross-scenario generalization.

Technology Category

• Machine Learning: Time-Series/Data Streams
• Intelligent Robots: State Estimation
• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

Irregularly sampled time series (ISTS), characterized by non-uniform time intervals with natural missingness, are prevalent in real-world applications. Existing approaches for ISTS modeling primarily rely on observed values to impute unobserved ones or infer latent dynamics. However, these methods overlook a critical source of learning signal: the reconstruction error inherently produced during model training. Such error implicitly reflects how well a model captures the underlying data structure and can serve as an informative proxy for unobserved values. To exploit this insight, we propose iTimER, a simple yet effective self-supervised pre-training framework for ISTS representation learning. iTimER models the distribution of reconstruction errors over observed values and generates pseudo-observations for unobserved timestamps through a mixup strategy between sampled errors and the last available observations. This transforms unobserved timestamps into noise-aware training targets, enabling meaningful reconstruction signals. A Wasserstein metric aligns reconstruction error distributions between observed and pseudo-observed regions, while a contrastive learning objective enhances the discriminability of learned representations. Extensive experiments on classification, interpolation, and forecasting tasks demonstrate that iTimER consistently outperforms state-of-the-art methods under the ISTS setting.

Problem

• Learning representations for irregularly sampled time series
• Exploiting reconstruction errors as learning signals
• Generating pseudo-observations for unobserved timestamps

Innovation

• Reconstruction error guides time series representation learning
• Mixup strategy generates pseudo-observations from sampled errors
• Wasserstein metric aligns error distributions for training