Worst-case low-rank approximations

2026-03-11

AI Summary

Standard principal component analysis (PCA) suffers from unreliable worst-case performance under distribution shifts across heterogeneous domains, such as different hospitals or time periods. To address this, this work proposes a worst-case PCA (wcPCA) framework that enhances the robustness of low-rank approximations by optimizing for worst-case rather than average performance over all possible target domains within the convex hull of source-domain covariances. The authors introduce novel estimators—including norm-minPCA and norm-maxregret—and extend worst-case optimization to inductive matrix completion for the first time. Theoretical analysis establishes statistical consistency and asymptotic worst-case optimality of the proposed approach. Empirical results demonstrate that wcPCA significantly improves worst-case performance on both simulated and real-world ecosystem–atmosphere flux data, with only a minor sacrifice in average performance.

Technology Category

• Machine Learning: Matrix & Tensor Methods
• Search and Optimization: Non-convex Optimization
• Reasoning under Uncertainty: Stochastic Optimization

Application Category

• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods
• Security and Privacy: Large-scale security measurements

Abstract

Real-world data in health, economics, and environmental sciences are often collected across heterogeneous domains (such as hospitals, regions, or time periods). In such settings, distributional shifts can make standard PCA unreliable, in that, for example, the leading principal components may explain substantially less variance in unseen domains than in the training domains. Existing approaches (such as FairPCA) have proposed to consider worst-case (rather than average) performance across multiple domains. This work develops a unified framework, called wcPCA, applies it to other objectives (resulting in the novel estimators such as norm-minPCA and norm-maxregret, which are better suited for applications with heterogeneous total variance) and analyzes their relationship. We prove that for all objectives, the estimators are worst-case optimal not only over the observed source domains but also over all target domains whose covariance lies in the convex hull of the (possibly normalized) source covariances. We establish consistency and asymptotic worst-case guarantees of empirical estimators. We extend our methodology to matrix completion, another problem that makes use of low-rank approximations, and prove approximate worst-case optimality for inductive matrix completion. Simulations and two real-world applications on ecosystem-atmosphere fluxes demonstrate marked improvements in worst-case performance, with only minor losses in average performance.

Problem

• low-rank approximation
• distributional shift
• worst-case performance
• heterogeneous domains
• principal component analysis

Innovation

• worst-case PCA
• domain heterogeneity
• convex hull of covariances
• low-rank approximation
• inductive matrix completion