Consistent Multigroup Low-Rank Approximation

2025-03-27

AI Summary

This paper introduces the multi-group consensus low-rank approximation problem: minimizing the maximum reconstruction error across groups—thereby ensuring group fairness—while enforcing nestedness, i.e., the first $d$ basis vectors must constitute the optimal $d$-dimensional solution for all $d$. This framework strictly generalizes classical single-group SVD and is the first to jointly model nestedness and min-max fairness. Methodologically, we propose an iterative greedy algorithm, where each iteration computes an optimal rank-1 orthogonal projection via either primal-dual optimization or semidefinite programming under orthogonality constraints; we provide theoretical guarantees on convergence and derive an approximation ratio bound. Experiments on multi-group PCA tasks demonstrate that our method significantly outperforms existing fair PCA approaches, achieving both theoretical rigor and empirical effectiveness.

Technology Category

• Machine Learning: Learning Preferences or Rankings
• Computer Vision: Bias, Fairness & Privacy
• Search and Optimization: Non-convex Optimization

Application Category

• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Economics, Online Markets and Human Computation: Fairness, privacy, and diversity in economic environments

Abstract

We consider the problem of consistent low-rank approximation for multigroup data: we ask for a sequence of $k$ basis vectors such that projecting the data onto their spanned subspace treats all groups as equally as possible, by minimizing the maximum error among the groups. Additionally, we require that the sequence of basis vectors satisfies the natural consistency property: when looking for the best $k$ vectors, the first $d<k$ vectors are the best possible solution to the problem of finding $d$ basis vectors. Thus, this multigroup low-rank approximation method naturally generalizes svd and reduces to svd for data with a single group. We give an iterative algorithm for this task that sequentially adds to the basis the vector that gives the best rank$-1$ projection according to the min-max criterion, and then projects the data onto the orthogonal complement of that vector. For finding the best rank$-1$ projection, we use primal-dual approaches or semidefinite programming. We analyze the theoretical properties of the algorithms and demonstrate empirically that the proposed methods compare favorably to existing methods for multigroup (or fair) PCA.

Problem

• Develop consistent low-rank approximation for multigroup data
• Minimize maximum error among groups in subspace projection
• Ensure basis vectors satisfy natural consistency property

Innovation

• Multigroup low-rank approximation with min-max error
• Iterative algorithm for consistent basis vectors
• Primal-dual and semidefinite programming for rank-1 projection