🤖 AI Summary
Time-series foundation models remain underexplored; existing approaches are largely task-specific, rely heavily on hand-crafted features, and lack cross-task transferable representations. To address this, we propose CHARM—the first foundation embedding model for multivariate time series—introducing channel-wise textual descriptions as domain priors to construct a channel-order-agnostic semantic-temporal joint embedding architecture. We design a self-supervised training framework based on Joint Embedding Predictive Architecture (JEPA), integrating invariance regularization and interpretability-aware loss, alongside a novel time-series-specific data augmentation strategy. With only 7 million parameters, CHARM achieves state-of-the-art performance across diverse downstream tasks—including classification, anomaly detection, and forecasting—establishing a new benchmark for multivariate time-series representation learning.
📝 Abstract
Traditional time series models are task-specific and often depend on dataset-specific training and extensive feature engineering. While Transformer-based architectures have improved scalability, foundation models, commonplace in text, vision, and audio, remain under-explored for time series and are largely restricted to forecasting. We introduce $ extbf{CHARM}$, a foundation embedding model for multivariate time series that learns shared, transferable, and domain-aware representations. To address the unique difficulties of time series foundation learning, $ extbf{CHARM}$ incorporates architectural innovations that integrate channel-level textual descriptions while remaining invariant to channel order. The model is trained using a Joint Embedding Predictive Architecture (JEPA), with novel augmentation schemes and a loss function designed to improve interpretability and training stability. Our $7$M-parameter model achieves state-of-the-art performance across diverse downstream tasks, setting a new benchmark for time series representation learning.