Tiny-TSM: Efficiently Training a Lightweight SOTA Time Series Foundation Model

2025-11-24

AI Summary

To address the low training efficiency, reliance on complex hyperparameter tuning, or large-scale architectures in lightweight time-series foundation models, this paper proposes SynthTS—a synthetic data generation and augmentation pipeline integrated with a causal input normalization mechanism. This combination enables, for the first time, rapid convergence and state-of-the-art (SOTA) performance for small models on dense forecasting tasks. Our 23M-parameter architecture trains efficiently on a single A100 GPU using next-token prediction loss and a hyperparameter-free training protocol. In medium- to long-term forecasting, it achieves uniformly lower MSE than existing methods—matching the performance of large industrial models—while remaining highly competitive in short-term forecasting. The core contribution lies in enabling efficient, low-cost, high-performance time-series foundation modeling under resource constraints, without neural architecture search or manual hyperparameter optimization.

Technology Category

• Machine Learning: Time-Series/Data Streams
• Natural Language Processing: Code Generation / Program Synthesis from Natural Language
• Humans and AI: Other Foundations of Human Computation & AI

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Web Mining and Content Analysis: Large pretrained models with web data
• Search and Retrieval-Augmented AI: Large language models for search

Abstract

We present Tiny-TSM, a time series foundation model characterized by small scale, economical training, and state-of-the-art performance. It comprises 23M total parameters, trained on a single A100 GPU in less than a week using a new synthetic data generation and data augmentation pipeline (SynthTS). Without any neural architecture search, hyperparameter tuning, or scaling up model size, Tiny-TSM achieves state-of-the-art performance on a wide range of time series benchmark datasets, often outperforming much larger models and even matching the performance of much larger, industrial-scale, likely highly tuned foundation models. Specifically, Tiny-TSM outperforms all other time series foundation models we evaluated on medium- and long-term forecasting tasks under MSE loss, while short-term accuracy is still competitive with state-of-the-art models. We also introduce a causal input normalization scheme that enables time series models to be trained with dense next-token prediction loss, significantly accelerating convergence speed and reducing training time. All experiments were conducted on a single A100 GPU, illustrating the practicality of the proposed approach in a resource-constrained setting.

Problem

• Developing lightweight time series model with efficient training requirements
• Achieving state-of-the-art performance without architecture search or scaling
• Enabling effective training in resource-constrained environments

Innovation

• Uses synthetic data generation pipeline for training
• Employs causal input normalization for faster convergence
• Achieves SOTA with small model on single GPU