A Scalable Factorization Approach for High-Order Structured Tensor Recovery

2025-06-19

AI Summary

High-order tensor recovery suffers from severe non-convexity due to multiplicative coupling among factors in structural tensor models. Method: This paper proposes an orthogonal factorization framework based on Stiefel manifold optimization, bypassing conventional alternating minimization and instead performing Riemannian gradient descent directly on the manifold. It establishes, for the first time, a unified Riemannian regularity condition applicable to multiple decomposition formats—including Tucker and tensor-train (TT) decompositions. Contribution/Results: Theoretically, the algorithm converges linearly to the ground-truth tensor, with convergence rate and initialization requirements scaling only polynomially in the tensor order (N)—overcoming prior exponential dependencies. Empirically, the framework significantly improves computational scalability and recovery accuracy in high-order settings.

Technology Category

• Machine Learning: Matrix & Tensor Methods
• Search and Optimization: Non-convex Optimization
• Reasoning under Uncertainty: Stochastic Optimization

Application Category

• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Tensor decompositions, which represent an $N$-order tensor using approximately $N$ factors of much smaller dimensions, can significantly reduce the number of parameters. This is particularly beneficial for high-order tensors, as the number of entries in a tensor grows exponentially with the order. Consequently, they are widely used in signal recovery and data analysis across domains such as signal processing, machine learning, and quantum physics. A computationally and memory-efficient approach to these problems is to optimize directly over the factors using local search algorithms such as gradient descent, a strategy known as the factorization approach in matrix and tensor optimization. However, the resulting optimization problems are highly nonconvex due to the multiplicative interactions between factors, posing significant challenges for convergence analysis and recovery guarantees. In this paper, we present a unified framework for the factorization approach to solving various tensor decomposition problems. Specifically, by leveraging the canonical form of tensor decompositions--where most factors are constrained to be orthonormal to mitigate scaling ambiguity--we apply Riemannian gradient descent (RGD) to optimize these orthonormal factors on the Stiefel manifold. Under a mild condition on the loss function, we establish a Riemannian regularity condition for the factorized objective and prove that RGD converges to the ground-truth tensor at a linear rate when properly initialized. Notably, both the initialization requirement and the convergence rate scale polynomially rather than exponentially with $N$, improving upon existing results for Tucker and tensor-train format tensors.

Problem

• Develop scalable tensor factorization for high-order data recovery
• Address nonconvex optimization challenges in tensor decomposition
• Ensure polynomial scalability with tensor order for convergence

Innovation

• Scalable factorization for high-order tensor recovery
• Riemannian gradient descent on Stiefel manifold
• Linear convergence with polynomial scaling