A Decomposition Framework for Certifiably Optimal Orthogonal Sparse PCA

2026-03-01

AI Summary

Existing sparse principal component analysis (SPCA) methods struggle to simultaneously achieve sparsity, orthogonality, and global optimality. This work proposes the GS-SPCA algorithm, which, for the first time, delivers a certifiably globally optimal solution under strict ℓ₀ sparsity constraints while guaranteeing orthogonality among principal components. The method integrates Gram–Schmidt orthogonalization, branch-and-bound optimization, and a threshold-based block partitioning with block-diagonal approximation of the covariance matrix to construct an efficient decomposition framework. Furthermore, it introduces an ε-optimal acceleration strategy that substantially enhances computational efficiency in multi-component settings while maintaining controllable solution accuracy.

Technology Category

• Constraint Satisfaction and Optimization: Distributed CSP/Optimization
• Search and Optimization: Non-convex Optimization
• Machine Learning: Optimization

Application Category

• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Search and Retrieval-Augmented AI: Efficiency and scalability of Web search engines
• Security and Privacy: Large-scale security measurements

Abstract

Sparse Principal Component Analysis (SPCA) is an important technique for high-dimensional data analysis, improving interpretability by imposing sparsity on principal components. However, existing methods often fail to simultaneously guarantee sparsity, orthogonality, and optimality of the principal components. To address this challenge, this work introduces a novel Sparse Principal Component Analysis (SPCA) algorithm called \textsc{GS-SPCA} (SPCA with Gram-Schmidt Orthogonalization), which simultaneously enforces sparsity, orthogonality, and optimality. However, the original GS-SPCA algorithm is computationally expensive due to the inherent $\ell_0$-norm constraint. To address this issue, we propose two acceleration strategies: First, we combine \textbf{Branch-and-Bound} with the GS-SPCA algorithm. By incorporating this strategy, we are able to obtain $\varepsilon$-optimal solutions with a trade-off between precision and efficiency, significantly improving computational speed. Second, we propose a \textbf{decomposition framework} for efficiently solving \textbf{multiple} principal components. This framework approximates the covariance matrix using a block-diagonal matrix through a thresholding method, reducing the original SPCA problem to a set of block-wise subproblems on approximately block-diagonal matrices.

Problem

• Sparse PCA
• orthogonality
• sparsity
• optimality
• high-dimensional data analysis

Innovation

• Sparse PCA
• Gram-Schmidt Orthogonalization
• Branch-and-Bound
• Decomposition Framework
• Block-diagonal Approximation