HEPA: A Self-Supervised Horizon-Conditioned Event Predictive Architecture for Time Series

2026-05-11

AI Summary

This work addresses the challenge of predicting rare critical events—such as equipment failures or cardiac arrhythmias—in multivariate time series, where scarce annotations hinder model performance. To overcome this limitation, the authors propose a causal Transformer-based self-supervised learning framework that leverages a Joint-Embedding Predictive Architecture (JEPA) to pretrain future representations. After freezing the encoder, only a lightweight prediction head is fine-tuned to output a monotonic survival cumulative distribution function estimating event occurrence time. The proposed general-purpose architecture unifies 14 distinct event prediction tasks across 11 domains and consistently outperforms state-of-the-art models—including PatchTST and iTransformer—on at least 10 benchmarks, while reducing both hyperparameter tuning effort and required labeled data by an order of magnitude.

Abstract

Critical events in multivariate time series, from turbine failures to cardiac arrhythmias, demand accurate prediction, yet labeled data is scarce because such events are rare and costly to annotate. We introduce HEPA (Horizon-conditioned Event Predictive Architecture), built on two key principles. First, a causal Transformer encoder is pretrained via a Joint-Embedding Predictive Architecture (JEPA): a horizon-conditioned predictor learns to forecast future representations rather than future values, forcing the encoder to capture predictable temporal dynamics from unlabeled data alone. Second, we freeze the encoder and finetune only the predictor toward the target event, producing a monotonic survival cumulative distribution function (CDF) over horizons. With fixed architecture and optimiser hyperparameters across all benchmarks, HEPA handles water contamination, cyberattack detection, volatility regimes, and eight further event types across 11 domains, exceeding leading time-series architectures including PatchTST, iTransformer, MAE, and Chronos-2 on at least 10 of 14 benchmarks, with an order of magnitude fewer tuned parameters and, on lifecycle datasets, an order of magnitude less labeled data.

Problem

• time series
• event prediction
• label scarcity
• multivariate time series
• critical events

Innovation

• self-supervised learning
• horizon-conditioned prediction
• Joint-Embedding Predictive Architecture (JEPA)
• survival CDF
• causal Transformer