A Comparative Study on How Data Normalization Affects Zero-Shot Generalization in Time Series Foundation Models

2025-12-02

AI Summary

Scale heterogeneity and non-stationarity in cross-domain and multivariate time series severely impair the zero-shot generalization of time-series foundation models (TSFMs). Method: This work systematically investigates the impact of input normalization on TSFM zero-shot forecasting performance, evaluating diverse normalization strategies across four architecturally distinct TSFMs using zero-shot MASE. Contribution/Results: We find that REVIN—a lightweight, plug-and-play normalization—significantly improves zero-shot accuracy without dataset-level preprocessing: it reduces MASE by 89% over the unnormalized baseline, outperforms alternative normalization methods by 44%, and achieves the state-of-the-art zero-shot MASE of 0.84. Crucially, REVIN’s efficacy is shown to depend critically on tight coupling between model architecture and optimization objective. This study provides the first systematic evidence that lightweight, input-level normalization is a key enabler for robust zero-shot generalization in TSFMs.

Technology Category

• Machine Learning: Time-Series/Data Streams
• Computer Vision: Other Foundations of Computer Vision
• Natural Language Processing: Safety and Robustness

Application Category

• Search and Retrieval-Augmented AI: Vertical and domain-specific search
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Web Mining and Content Analysis: Normalization, clustering, classification, and summarization of Web text

Abstract

We investigate input normalization methods for Time-Series Foundation Models (TSFMs). While normalization is well-studied in dataset-specific time-series models, it remains overlooked in TSFMs where generalization is critical. Time-series data, unlike text or images, exhibits significant scale variation across domains and channels, coupled with non-stationarity, can undermine TSFM performance regardless of architectural complexity. Through systematic evaluation across four architecturally diverse TSFMs, we empirically establish REVIN as the most efficient approach, reducing zero-shot MASE by 89% relative to an un-normalized baseline and by 44% versus other normalization methods, while matching the best in-domain accuracy (0.84 MASE) without any dataset-level preprocessing -- yielding the highest accuracy-efficiency trade-off. Yet its effect utilization depends on architectural design choices and optimization objective, particularly with respect to training loss scale sensitivity and model type (probabilistic, point-forecast, or LLM-based models).

Problem

• Investigates normalization's impact on zero-shot generalization in time-series foundation models
• Addresses scale variation and non-stationarity in time-series data across domains
• Evaluates REVIN's efficiency in improving accuracy without dataset preprocessing

Innovation

• REVIN normalization improves zero-shot generalization
• Reduces zero-shot error by 89% versus un-normalized baseline
• Balances accuracy and efficiency without dataset preprocessing