AdaPTS: Adapting Univariate Foundation Models to Probabilistic Multivariate Time Series Forecasting

2025-02-14

AI Summary

This paper addresses the challenge of adapting pretrained univariate time-series foundation models to multivariate probabilistic forecasting. We propose a lightweight adapter-driven transfer framework that systematically introduces adapter mechanisms into time-series foundation models for the first time. Our method enables cross-dimensional feature mapping via latent-space projection and integrates dimension-wise independent modeling with joint optimization. Uncertainty quantification is supported by a partially randomized Bayesian neural network. Key contributions include a modular, scalable zero-shot transfer framework for multivariate forecasting that jointly captures inter-variable dependencies and ensures predictive calibration. Extensive experiments on synthetic and real-world benchmarks demonstrate consistent improvements: average MAE decreases by 18.7% and negative log-likelihood (NLL) improves by 23.4% over strong baselines. The implementation is publicly available.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Reasoning under Uncertainty: Relational Probabilistic Models
• Cognitive Modeling & Cognitive Systems: Adaptive Behavior

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Web Mining and Content Analysis: Large pretrained models with web data
• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization

Abstract

Pre-trained foundation models (FMs) have shown exceptional performance in univariate time series forecasting tasks. However, several practical challenges persist, including managing intricate dependencies among features and quantifying uncertainty in predictions. This study aims to tackle these critical limitations by introducing adapters; feature-space transformations that facilitate the effective use of pre-trained univariate time series FMs for multivariate tasks. Adapters operate by projecting multivariate inputs into a suitable latent space and applying the FM independently to each dimension. Inspired by the literature on representation learning and partially stochastic Bayesian neural networks, we present a range of adapters and optimization/inference strategies. Experiments conducted on both synthetic and real-world datasets confirm the efficacy of adapters, demonstrating substantial enhancements in forecasting accuracy and uncertainty quantification compared to baseline methods. Our framework, AdaPTS, positions adapters as a modular, scalable, and effective solution for leveraging time series FMs in multivariate contexts, thereby promoting their wider adoption in real-world applications. We release the code at https://github.com/abenechehab/AdaPTS.

Problem

• Adapting univariate models to multivariate forecasting
• Managing feature dependencies in time series
• Quantifying uncertainty in probabilistic forecasts

Innovation

• Adapters for multivariate time series
• Latent space projection technique
• Enhanced forecasting accuracy and uncertainty