Generalized Temporal Tensor Decomposition with Rank-revealing Latent-ODE

2025-02-10

AI Summary

This work addresses the decomposition of multimodal continuous-indexed tensors (e.g., spatiotemporal climate data), proposing the first unified tensor decomposition framework capable of modeling arbitrary continuous modes—such as time and space—natively. Methodologically, it introduces a rank-revealing Gaussian–Gamma prior to jointly learn factor trajectories and intrinsic tensor rank; develops a sampling-free variational inference framework that integrates Fourier feature encoding with neural ordinary differential equations for continuous-domain modeling, and derives analytical optimization of the evidence lower bound (ELBO). Unlike existing discretization-based or fixed-rank approaches, our method is the first to enable automatic rank determination across multimodal continuous indices. Experiments on synthetic and real-world datasets demonstrate significant improvements in prediction accuracy and noise robustness. The framework establishes a novel paradigm for modeling high-dimensional continuous sensing data.

Technology Category

• Machine Learning: Matrix & Tensor Methods
• Reasoning under Uncertainty: Relational Probabilistic Models
• Computer Vision: Multi-modal Vision

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Tensor decomposition is a fundamental tool for analyzing multi-dimensional data by learning low-rank factors to represent high-order interactions. While recent works on temporal tensor decomposition have made significant progress by incorporating continuous timestamps in latent factors, they still struggle with general tensor data with continuous indexes not only in the temporal mode but also in other modes, such as spatial coordinates in climate data. Additionally, the problem of determining the tensor rank remains largely unexplored in temporal tensor models. To address these limitations, we propose underline{G}eneralized temporal tensor decomposition with underline{R}ank-runderline{E}vealing latenunderline{T}-ODE (GRET). Our approach encodes continuous spatial indexes as learnable Fourier features and employs neural ODEs in latent space to learn the temporal trajectories of factors. To automatically reveal the rank of temporal tensors, we introduce a rank-revealing Gaussian-Gamma prior over the factor trajectories. We develop an efficient variational inference scheme with an analytical evidence lower bound, enabling sampling-free optimization. Through extensive experiments on both synthetic and real-world datasets, we demonstrate that GRET not only reveals the underlying ranks of temporal tensors but also significantly outperforms existing methods in prediction performance and robustness against noise.

Problem

• Generalized tensor decomposition with continuous indexes
• Rank determination in temporal tensor models
• Enhancing prediction performance and noise robustness

Innovation

• Fourier features encoding
• Neural ODEs in latent space
• Rank-revealing Gaussian-Gamma prior