UniCA: Adapting Time Series Foundation Model to General Covariate-Aware Forecasting

2025-06-27

AI Summary

Existing time-series foundation models (TSFMs) are primarily pretrained on real-valued univariate or multivariate time series and struggle to effectively incorporate task-specific heterogeneous covariates—such as categorical variables, text, or images. To address this limitation, we propose UniCA, a Unified Covariate Adaptation framework—the first general-purpose approach enabling TSFMs to seamlessly integrate multimodal, heterogeneous covariates. UniCA comprises two core components: (1) a covariate homogenization module that maps diverse covariates into a unified latent space temporally aligned with the input time series; and (2) a lightweight attention-based fusion mechanism that dynamically integrates covariate information without modifying pretrained TSFM weights. Crucially, UniCA preserves the original TSFM’s generalization capability while substantially improving downstream forecasting performance. Extensive experiments across multiple single- and multimodal covariate benchmarks demonstrate state-of-the-art results, validating UniCA’s effectiveness, robustness, and broad applicability.

Technology Category

• Machine Learning: Multimodal Learning
• Computer Vision: Multi-modal Vision
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Graph Algorithms and Modeling for the Web: Algorithms and analysis for heterogeneous, signed, attributed, multi-relational, temporal, higher-order, and annotated Web-related graphs
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Web Mining and Content Analysis: Large pretrained models with web data

Abstract

Time Series Foundation Models (TSFMs) have achieved remarkable success through large-scale pretraining. However, their design primarily targets real-valued series, limiting their ability to handle general forecasting tasks involving diverse and often heterogeneous covariates--such as categorical variables and multimodal data (e.g., images, text)--which are typically task-specific and difficult to leverage during pretraining. To address this gap, we propose Unified Covariate Adaptation (UniCA), a framework to bridge TSFMs with general covariate-aware forecasting. UniCA first performs covariate homogenization to transform heterogeneous covariates into high-level homogeneous series representations and then fuses them via a unified attention-based fusion mechanism. UniCA is compatible and universal for adaptation with both homogeneous and heterogeneous covariates, incorporating extra covariate information while preserving the generalization ability of TSFMs.Extensive experiments on multiple unimodal and multimodal covariate-aware forecasting benchmarks demonstrate the superiority of UniCA, highlighting the promise of covariate-aware TSFM adaptation in real-world forecasting scenarios. Codes are released on https://github.com/hanlu-nju/UniCA.

Problem

• Adapts time series models to handle diverse covariates
• Transforms heterogeneous data into unified representations
• Enhances forecasting with multimodal covariate integration

Innovation

• Adapts time series models for diverse covariates
• Homogenizes covariates into unified representations
• Uses attention-based fusion for covariate integration