π€ AI Summary
This work addresses the challenges of statistical inconsistency, high communication overhead, and privacy concerns in traditional principal component analysis (PCA) when applied to high-dimensional distributed data. The authors propose a novel distributed PCA framework that incorporates wavelet-based sparsification, leveraging wavelet transforms to obtain sparse representations of variables. By integrating this sparsification with a distributed optimization algorithm, the method efficiently estimates the global shared subspace without requiring raw data aggregation. The approach preserves statistical consistency while substantially reducing communication costs. Experimental results demonstrate that, for dimensions \(d \geq 152\), the proposed method consistently outperforms existing approaches in both estimation accuracy and the number of transmitted coefficients.
π Abstract
The large volume of data and concerns about data privacy have motivated the development of techniques for distributed data, a problem also known as federated learning. In this scenario, sub-samples of the data are divided across different machines, and statistics must be computed over that data without direct access to the full sample. Johnstone & Lu (2009, JASA) show that principal component analysis (PCA) is statistically inconsistent in the high-dimensional regime, and propose a way to recover consistency through wavelet-based sparsification and variable selection. Fan et al. (2019, AoS) show a way to perform this same estimation -- specifically, to estimate the eigenspace that would be obtained if all the data were pooled together, even though it remains effectively distributed -- without addressing the high-dimensional regime. This work incorporates the wavelet-based sparsification of Johnstone & Lu (2009) into the distributed PCA framework of Fan et al. (2019), aiming to reduce communication cost without compromising the quality of the eigenspace estimation. Simulations across $d \in [52, 5000]$ show that the proposed method overtakes Fan et al. (2019) in estimation error beyond a clear dimensional threshold ($d \geq 152$ for $Ξ»=25$, $d \geq 252$ for $Ξ»=50$), while transmitting systematically fewer coefficients throughout the entire range studied. This study was financed by the Sao Paulo Research Foundation (FAPESP), Brazil. Process Number #2023/02538-0 and Number #2025/21250-2.