A Tight Lower Bound for the Approximation Guarantee of Higher-Order Singular Value Decomposition

2025-08-08

AI Summary

This work addresses the long-standing open problem of tightness of approximation guarantees for higher-order singular value decomposition (HOSVD) and its variants—sequential truncated HOSVD (ST-HOSVD) and the higher-order orthogonal iteration (HOOI). We construct, for the first time, a family of pathological tensors that rigorously demonstrates that, for any ε > 0, all three algorithms achieve an approximation ratio of *N* / (1 + ε) in the worst case, matching the previously known upper bounds. This establishes the exact tightness of their approximation ratios, confirming that classical worst-case analyses are optimal and cannot be improved. Methodologically, our approach integrates tensor-based counterexample construction, multilinear algebraic analysis, and low-rank approximation theory. The result provides the first tight characterization of the theoretical limits of these widely used high-dimensional data dimensionality reduction algorithms.

Technology Category

• Machine Learning: Matrix & Tensor Methods
• Search and Optimization: Non-convex Optimization
• Constraint Satisfaction and Optimization: Other Foundations of Constraint Satisfaction

Application Category

• Graph Algorithms and Modeling for the Web: Algorithms and analysis for heterogeneous, signed, attributed, multi-relational, temporal, higher-order, and annotated Web-related graphs
• Systems and Infrastructure for Web, Mobile and WoT: Experiences and lessons learnt from Web-based algorithms and system deployments
• Web Mining and Content Analysis: Web data quality in the era of algorithmically-generated content

Abstract

We prove that the classic approximation guarantee for the higher-order singular value decomposition (HOSVD) is tight by constructing a tensor for which HOSVD achieves an approximation ratio of $N/(1+varepsilon)$, for any $varepsilon > 0$. This matches the upper bound of De Lathauwer et al. (2000a) and shows that the approximation ratio of HOSVD cannot be improved. Using a more advanced construction, we also prove that the approximation guarantees for the ST-HOSVD algorithm of Vannieuwenhoven et al. (2012) and higher-order orthogonal iteration (HOOI) of De Lathauwer et al. (2000b) are tight by showing that they can achieve their worst-case approximation ratio of $N / (1 + varepsilon)$, for any $varepsilon > 0$.

Problem

• Proves tightness of HOSVD approximation guarantee
• Shows ST-HOSVD and HOOI worst-case ratios are tight
• Constructs tensors achieving theoretical lower bounds

Innovation

• Proves HOSVD approximation ratio is tight
• Constructs tensor achieving worst-case ratio
• Extends tightness proof to ST-HOSVD and HOOI