Guaranteed Nonconvex Factorization Approach for Tensor Train Recovery

2024-01-05

arXiv.org

AI Summary

This paper addresses the robust recovery of high-order tensors under low-rank tensor train (TT) structure. We propose a nonconvex Riemannian optimization method based on left-orthogonal TT decomposition. The algorithm performs Riemannian gradient descent on the Stiefel manifold, coupled with spectral initialization and restricted isometry property (RIP) analysis, ensuring robust recovery from linear measurements corrupted by Gaussian noise. Theoretically, we establish the first rigorous convergence guarantee for TT factorization: the algorithm achieves linear convergence, with rate degrading only linearly in tensor order; under noise, the reconstruction error grows polynomially in the noise level. Experiments demonstrate that our method significantly outperforms existing convex relaxation approaches in both convergence speed and recovery accuracy.

Technology Category

• Machine Learning: Matrix & Tensor Methods
• Search and Optimization: Non-convex Optimization
• Reasoning under Uncertainty: Stochastic Optimization

Application Category

• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs
• Systems and Infrastructure for Web, Mobile and WoT: Experiences and lessons learnt from Web-based algorithms and system deployments
• Security and Privacy: Large-scale security measurements

Abstract

In this paper, we provide the first convergence guarantee for the factorization approach. Specifically, to avoid the scaling ambiguity and to facilitate theoretical analysis, we optimize over the so-called left-orthogonal TT format which enforces orthonormality among most of the factors. To ensure the orthonormal structure, we utilize the Riemannian gradient descent (RGD) for optimizing those factors over the Stiefel manifold. We first delve into the TT factorization problem and establish the local linear convergence of RGD. Notably, the rate of convergence only experiences a linear decline as the tensor order increases. We then study the sensing problem that aims to recover a TT format tensor from linear measurements. Assuming the sensing operator satisfies the restricted isometry property (RIP), we show that with a proper initialization, which could be obtained through spectral initialization, RGD also converges to the ground-truth tensor at a linear rate. Furthermore, we expand our analysis to encompass scenarios involving Gaussian noise in the measurements. We prove that RGD can reliably recover the ground truth at a linear rate, with the recovery error exhibiting only polynomial growth in relation to the tensor order. We conduct various experiments to validate our theoretical findings.

Problem

• Guaranteeing convergence for tensor train factorization approach
• Recovering TT format tensors from linear measurements with RIP
• Handling Gaussian noise scenarios with reliable recovery guarantees

Innovation

• Riemannian gradient descent on Stiefel manifold
• Left-orthogonal TT format optimization
• Spectral initialization with RIP guarantees