SeqFusion: Sequential Fusion of Pre-Trained Models for Zero-Shot Time-Series Forecasting

2025-03-04

AI Summary

Zero-shot time-series forecasting faces dual challenges—absence of target-domain training data and stringent requirements for data privacy preservation. Method: We propose a fine-tuning-free, representation-alignment-based framework for dynamic fusion of multiple pre-trained models (PTMs). First, heterogeneous time series are mapped into a shared representation space. Then, model-level feature distance metrics enable adaptive, sequential selection and invocation of specialized PTMs. Finally, their predictions are weighted and fused. Contribution/Results: Our work introduces the first “sequential PTM selection–inference–fusion” mechanism, eliminating reliance on diverse private pre-training datasets. Experiments demonstrate state-of-the-art (SOTA) accuracy under zero-shot settings, significantly reducing access to sensitive historical data while jointly optimizing predictive performance, cross-domain generalizability, and privacy protection.

Technology Category

• Machine Learning: Time-Series/Data Streams
• Natural Language Processing: Language Grounding & Multi-modal NLP
• Computer Vision: Diffusion Models for Vision

Application Category

• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Unlike traditional time-series forecasting methods that require extensive in-task data for training, zero-shot forecasting can directly predict future values given a target time series without additional training data. Current zero-shot approaches primarily rely on pre-trained generalized models, with their performance often depending on the variety and relevance of the pre-training data, which can raise privacy concerns. Instead of collecting diverse pre-training data, we introduce SeqFusion in this work, a novel framework that collects and fuses diverse pre-trained models (PTMs) sequentially for zero-shot forecasting. Based on the specific temporal characteristics of the target time series, SeqFusion selects the most suitable PTMs from a batch of pre-collected PTMs, performs sequential predictions, and fuses all the predictions while using minimal data to protect privacy. Each of these PTMs specializes in different temporal patterns and forecasting tasks, allowing SeqFusion to select by measuring distances in a shared representation space of the target time series with each PTM. Experiments demonstrate that SeqFusion achieves competitive accuracy in zero-shot forecasting compared to state-of-the-art methods.

Problem

• Zero-shot time-series forecasting without additional training data
• Privacy concerns in using diverse pre-training data
• Sequential fusion of pre-trained models for accurate predictions

Innovation

• SeqFusion fuses diverse pre-trained models sequentially
• Selects PTMs based on target time-series characteristics
• Minimizes data usage to enhance privacy protection