StablePCA: Learning Shared Representations across Multiple Sources via Minimax Optimization

2025-05-02

AI Summary

To address pronounced batch effects, poor cross-source transferability, and insufficient fairness in multi-source high-dimensional data, this paper proposes a Distributionally Robust Principal Component Analysis (DR-PCA) framework. It is the first to integrate distributionally robust learning into PCA, formulating a convexified minimax optimization problem via Fantope relaxation and solving it with an optimistic gradient Mirror Prox algorithm featuring closed-form updates. Theoretically, the algorithm is proven to converge globally with an explicit convergence rate. Empirically, under finite-sample settings, DR-PCA significantly outperforms existing methods in dimensionality reduction accuracy, cross-dataset transfer performance, batch effect correction, and group fairness—while maintaining computational efficiency.

Technology Category

• Machine Learning: Dimensionality Reduction/Feature Selection
• Search and Optimization: Distributed Search
• Intelligent Robots: Learning & Optimization for ROB

Application Category

• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Responsible Web: Algorithmic accountability and transparency on the web

Abstract

When synthesizing multisource high-dimensional data, a key objective is to extract low-dimensional feature representations that effectively approximate the original features across different sources. Such general feature extraction facilitates the discovery of transferable knowledge, mitigates systematic biases such as batch effects, and promotes fairness. In this paper, we propose Stable Principal Component Analysis (StablePCA), a novel method for group distributionally robust learning of latent representations from high-dimensional multi-source data. A primary challenge in generalizing PCA to the multi-source regime lies in the nonconvexity of the fixed rank constraint, rendering the minimax optimization nonconvex. To address this challenge, we employ the Fantope relaxation, reformulating the problem as a convex minimax optimization, with the objective defined as the maximum loss across sources. To solve the relaxed formulation, we devise an optimistic-gradient Mirror Prox algorithm with explicit closed-form updates. Theoretically, we establish the global convergence of the Mirror Prox algorithm, with the convergence rate provided from the optimization perspective. Furthermore, we offer practical criteria to assess how closely the solution approximates the original nonconvex formulation. Through extensive numerical experiments, we demonstrate StablePCA's high accuracy and efficiency in extracting robust low-dimensional representations across various finite-sample scenarios.

Problem

• Extracting low-dimensional features from multi-source high-dimensional data
• Overcoming nonconvexity in multi-source PCA via minimax optimization
• Ensuring robust shared representations across diverse data sources

Innovation

• Uses Fantope relaxation for convex minimax optimization
• Employs optimistic-gradient Mirror Prox algorithm
• Extracts robust low-dimensional representations efficiently