Abstracted Shapes as Tokens - A Generalizable and Interpretable Model for Time-series Classification

2024-11-01

arXiv.org

AI Summary

Existing time-series classification models suffer from poor interpretability, weak cross-domain generalization, and inconsistent representation learning. To address these issues, we propose VQShape—the first vector quantization (VQ)-based foundation model for time series. VQShape introduces shape-level tokenization, mapping raw sequences into semantically meaningful, visually interpretable shape tokens that explicitly align latent representations with human-understandable temporal patterns. Our method integrates self-supervised pretraining with shape abstraction encoding. On multi-domain time-series classification benchmarks, VQShape matches or exceeds the performance of task-specific models. Notably, it achieves zero-shot cross-domain transfer to unseen datasets and domains—a first in time-series foundation modeling. Moreover, VQShape offers strong interpretability, computational efficiency, and broad applicability across diverse time-series tasks. To foster reproducibility and further research, we release our code and pretrained weights publicly.

Technology Category

• Machine Learning: Time-Series/Data Streams
• Computer Vision: Large Vision Models
• Natural Language Processing: (Large) Language Models

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Web Mining and Content Analysis: Large pretrained models with web data
• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization

Abstract

In time-series analysis, many recent works seek to provide a unified view and representation for time-series across multiple domains, leading to the development of foundation models for time-series data. Despite diverse modeling techniques, existing models are black boxes and fail to provide insights and explanations about their representations. In this paper, we present VQShape, a pre-trained, generalizable, and interpretable model for time-series representation learning and classification. By introducing a novel representation for time-series data, we forge a connection between the latent space of VQShape and shape-level features. Using vector quantization, we show that time-series from different domains can be described using a unified set of low-dimensional codes, where each code can be represented as an abstracted shape in the time domain. On classification tasks, we show that the representations of VQShape can be utilized to build interpretable classifiers, achieving comparable performance to specialist models. Additionally, in zero-shot learning, VQShape and its codebook can generalize to previously unseen datasets and domains that are not included in the pre-training process. The code and pre-trained weights are available at https://github.com/YunshiWen/VQShape.

Problem

• Time Series Classification
• Interpretability
• Efficiency

Innovation

• VQShape
• Vector Quantization
• Time Series Analysis