TiCT: A Synthetically Pre-Trained Foundation Model for Time Series Classification

2025-11-24

AI Summary

To address the high labeling cost in time-series classification and the lack of zero-shot generalization capability in existing large models, this paper introduces the first foundation model for universal time-series classification supporting in-context learning. Methodologically, it proposes bit-level label encoding and an output attention mechanism to enable zero-shot classification across arbitrary numbers of classes; designs a self-supervised pretraining framework leveraging Mixup and synthetic data augmentation to enhance generalization and robustness; and adopts a pure Transformer architecture—eliminating fine-tuning entirely and relying solely on in-context examples for inference. Evaluated on the UCR Archive benchmark, the model achieves zero-shot performance competitive with state-of-the-art supervised methods, thereby establishing, for the first time, the feasibility and effectiveness of foundation models in time-series classification.

Technology Category

• Machine Learning: Time-Series/Data Streams
• Computer Vision: Large Vision Models
• Natural Language Processing: (Large) Language Models

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Web Mining and Content Analysis: Large pretrained models with web data
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation

Abstract

The ubiquity of time series data creates a strong demand for general-purpose foundation models, yet developing them for classification remains a significant challenge, largely due to the high cost of labeled data. Foundation models capable of in-context learning (ICL) offer a powerful solution, adapting to new tasks with minimal examples and reducing the need for extensive retraining. However, prior work on large-scale time series models has predominantly focused on forecasting, leaving a critical gap for versatile, fine-tuning-free classification. To address this, we introduce TiCT (Time-series in-Context Transformer), a transformer-based model pre-trained exclusively on synthetic data to perform in-context classification. We make two primary technical contributions: 1) a novel architecture featuring a scalable bit-based label encoding and a special output attention mechanism to handle an arbitrary number of classes; and 2) a synthetic pre-training framework that combines a Mixup-inspired process with data augmentation to foster generalization and noise invariance. Extensive evaluations on the UCR Archive show that TiCT achieves competitive performance against state-of-the-art supervised methods. Crucially, this is accomplished using only in-context examples at inference time, without updating a single model weight.

Problem

• Developing foundation models for time series classification with limited labeled data
• Enabling in-context learning for time series classification without fine-tuning
• Creating synthetic pre-training methods for generalization in time series analysis

Innovation

• Pre-trained transformer model using synthetic data
• Scalable bit-based label encoding for arbitrary classes
• Mixup-inspired data augmentation for generalization enhancement