ViTime: A Visual Intelligence-Based Foundation Model for Time Series Forecasting

2024-07-10

arXiv.org

AI Summary

Time-series forecasting suffers from poor generalization and limited cross-task transferability. Method: This paper introduces the first vision-based foundation model for multi-domain time-series forecasting. It proposes a novel representation paradigm that maps time series into an image metric space and designs a Time-Series Visualization Transformer (ViT) architecture; introduces a realistic periodicity-trend joint synthesis strategy to enhance pattern robustness; and develops a no-imputation mechanism for modeling missing values. Contributions/Results: The model achieves 9–15% improvement over TimesFM under zero-shot transfer; surpasses fully supervised state-of-the-art (SOTA) methods using only 10% labeled data; demonstrates 20–30% superior robustness against perturbations; and attains SOTA performance across diverse real-world scenarios. By transcending conventional numerical fitting paradigms, it establishes a new foundation-model framework for general-purpose time-series forecasting.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Machine Learning: Time-Series/Data Streams
• Cognitive Modeling & Cognitive Systems: Other Foundations of Cognitive Modeling & Systems

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Search and Retrieval-Augmented AI: Vertical and domain-specific search
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

Time series forecasting (TSF) possesses great practical values in various fields, including power and energy, transportation, etc. TSF methods have been studied based on knowledge from classical statistics to modern deep learning. Yet, all of them were developed based on one fundamental concept, the numerical data fitting. Thus, the models developed have been long known for being problem-specific and lacking application generalizability. A TSF foundation model serving TSF tasks across different applications can reverse such an impression. The central question is then how to develop such a TSF foundation model. This paper offers a pioneering study in developing a TSF foundation model and proposes a vision intelligence-powered framework, ViTime, for the first time. In ViTime, a method synthesizing authentic time series periodic and trend patterns is developed to enrich sample pattern diversity. A deep architecture operating TSF in image metric space is designed to achieve significantly enhanced TSF generalizability. Extensive experiments demonstrate ViTime's SOTA performance across multiple settings. In zero-shot scenarios, ViTime outperforms TimesFM by 9-15%. With just 10% fine-tuning data, ViTime surpasses both foundation models and fully-supervised benchmarks trained on complete datasets, with this performance gap widening further at 100% fine-tuning. Additionally, ViTime exhibits exceptional robustness, handling missing data without imputation and outperforming TimesFM by 20-30% under various data perturbations.

Problem

• Develop a generalizable TSF foundation model
• Enhance TSF with visual intelligence framework
• Improve robustness and accuracy in forecasting

Innovation

• Vision intelligence-powered framework
• Deep architecture in image metric
• Synthesizing time series patterns