Tabby: An Open Pretraining Recipe for Time Series Foundation Models

2026-09-12

AI Summary

该研究发布了一个名为Tabby的时间序列基础模型，采用编码器-only patch Transformer架构，并结合多种数据集和训练方法，支持预测、分类和异常检测。

Abstract

In this report, we release Tabby, a long context probabilistic time series foundation model, together with a complete and open recipe of how it was built. Tabby adopts an encoder-only patch Transformer architecture and concentrates the contributions on the data and the training procedure. The pretraining corpus combines an extended real-world collection, GIFT-Eval-Pretrain+ and BLAST, with synthetic data from KernelSynth and CauKerV2, an online generator that composes temporal dynamics through randomly sampled structural causal models. Training couples a progressive convergence schedule, which yields reusable intermediate checkpoints, with a deep quantile supervision objective for intermediate layers. The resulting 145M parameter backbone supports contexts of up to 8,192 observations and serves forecasting, classification, and anomaly detection, while a prompt-tuning module further improves in-distribution forecasting performance with the pretrained weights frozen. Tabby achieves competitive zero-shot forecasting performance on GIFT-Eval and the out-of-distribution TIME benchmark, while the same pretrained backbone also supports classification on the UCR Archive and zero-shot anomaly detection on TSB-AD-U. We release training pipeline and model as open source at huawei-noah/trustworthyAI.

Problem

• time series
• foundation model
• forecasting
• classification
• anomaly detection

Innovation

• Patch Transformer
• progressive convergence schedule
• deep quantile supervision
• prompt-tuning module
• long context