SARMA: Scalable Low-Rank High-Dimensional Autoregressive Moving Averages via Tensor Decomposition

2024-05-01

AI Summary

Existing high-dimensional time series modeling is largely confined to finite-order VAR models, limiting their capacity to capture complex VARMA-type dynamics. Method: We propose SARMA, the first framework to incorporate infinite-order VARMA dynamics into high-dimensional settings, enabled by Tucker decomposition-based low-rank tensor parameterization. SARMA integrates a flexible temporal tensor structure, sparse factor loadings, and ℓ₁ regularization to support automatic variable selection and interpretable dynamic grouping; model estimation employs alternating optimization, while model selection is guided by information criteria. Contribution/Results: We establish statistical consistency of the estimator. Empirical evaluations demonstrate that SARMA significantly outperforms conventional VAR and VARMA models in forecasting accuracy, parameter efficiency, and interpretability.

Technology Category

• Machine Learning: Matrix & Tensor Methods
• Intelligent Robots: State Estimation
• Reasoning under Uncertainty: Relational Probabilistic Models

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization

Abstract

Existing models for high-dimensional time series are overwhelmingly developed within the finite-order vector autoregressive (VAR) framework. However, the more flexible vector autoregressive moving averages (VARMA) have been much less considered. This paper introduces a Tucker-low-rank framework to efficiently capture VARMA-type dynamics for high-dimensional time series, named the Scalable ARMA (SARMA) model. It generalizes the Tucker-low-rank finite-order VAR model to the infinite-order case via flexible parameterizations of the AR coefficient tensor along the temporal dimension. The resulting model enables dynamic factor extraction across response and predictor variables, facilitating interpretation of group patterns. Additionally, we consider sparsity assumptions on the factor loadings to accomplish automatic variable selection and greater estimation efficiency. Both rank-constrained and sparsity-inducing estimators are developed for the proposed model, along with algorithms and model selection methods. The validity of our theory and empirical advantages of our approach are confirmed by simulation studies and real data examples.

Problem

• Develop scalable low-rank VARMA model for high-dimensional time series
• Enable dynamic factor extraction for group pattern interpretation
• Incorporate sparsity for automatic variable selection and efficiency

Innovation

• Tucker-low-rank framework for VARMA dynamics
• Dynamic factor extraction across variables
• Sparsity-inducing estimators for efficiency