A Multi-Task Learning Approach to Linear Multivariate Forecasting

2025-02-05

AI Summary

Existing multivariate time series forecasting methods often neglect dynamic inter-variable dependencies, leading to modeling bias. To address this, we formulate multivariate forecasting as a systematic multi-task learning problem for the first time. Our approach introduces a gradient-geometry-based task partitioning and balancing mechanism: (i) task correlation is quantified via gradient angle analysis; (ii) correlation-driven hierarchical clustering groups variables into coherent tasks; and (iii) an error-adaptive gradient reweighting strategy ensures balanced optimization. We further propose MTLinear—a lightweight linear architecture that achieves strong expressiveness without sacrificing computational efficiency. Extensive experiments on multiple benchmark datasets demonstrate that our method consistently outperforms strong baselines—including Informer and Autoformer—in both accuracy and inference speed, achieving superior forecasting performance with significantly lower computational overhead. The implementation is publicly available.

Technology Category

• Machine Learning: Time-Series/Data Streams
• Planning, Routing, and Scheduling: Optimization of Spatio-temporal Systems
• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs

Abstract

Accurate forecasting of multivariate time series data is important in many engineering and scientific applications. Recent state-of-the-art works ignore the inter-relations between variates, using their model on each variate independently. This raises several research questions related to proper modeling of multivariate data. In this work, we propose to view multivariate forecasting as a multi-task learning problem, facilitating the analysis of forecasting by considering the angle between task gradients and their balance. To do so, we analyze linear models to characterize the behavior of tasks. Our analysis suggests that tasks can be defined by grouping similar variates together, which we achieve via a simple clustering that depends on correlation-based similarities. Moreover, to balance tasks, we scale gradients with respect to their prediction error. Then, each task is solved with a linear model within our MTLinear framework. We evaluate our approach on challenging benchmarks in comparison to strong baselines, and we show it obtains on-par or better results on multivariate forecasting problems. The implementation is available at: https://github.com/azencot-group/MTLinear

Problem

• Multivariate time series forecasting
• Multi-task learning approach
• Inter-relations between variates

Innovation

• Multi-task learning for multivariate forecasting
• Clustering variates by correlation-based similarities
• Scaling gradients based on prediction error