🤖 AI Summary
Existing principal component analysis (PCA) model selection methods lack statistical guarantees for determining the number of leading components under heteroscedastic noise—where observation-wise noise variances differ—in high-dimensional settings.
Method: We propose Signflip Parallel Analysis (Signflip PA), a novel parallel analysis method that generates an empirical null distribution via random sign flips and adaptively calibrates singular value thresholds.
Contribution/Results: Signflip PA is the first to integrate dimension-free operator norm bounds and large-deviation theory for eigenvalues of non-homogeneous matrices into PCA model selection, ensuring consistent factor recovery. We establish its theoretical consistency under a signal-plus-heteroscedastic-noise model. Empirical studies—including simulations and real-data analyses—demonstrate that Signflip PA significantly outperforms classical approaches such as scree plots and conventional parallel analysis, overcoming the fundamental limitation wherein heteroscedasticity causes traditional methods to fail.
📝 Abstract
Dimensionality reduction via PCA and factor analysis is an important tool of data analysis. A critical step is selecting the number of components. However, existing methods (such as the scree plot, likelihood ratio, parallel analysis, etc) do not have statistical guarantees in the increasingly common setting where the data are heterogeneous. There each noise entry can have a different distribution. To address this problem, we propose the Signflip Parallel Analysis (Signflip PA) method: it compares data singular values to those of "empirical null" data generated by flipping the sign of each entry randomly with probability one-half. We show that Signflip PA consistently selects factors above the noise level in high-dimensional signal-plus-noise models (including spiked models and factor models) under heterogeneous settings. Here classical parallel analysis is no longer effective. To do this, we propose to leverage recent breakthroughs in random matrix theory, such as dimension-free operator norm bounds [Latala et al, 2018, Inventiones Mathematicae], and large deviations for the top eigenvalues of nonhomogeneous matrices [Husson, 2020]. To our knowledge, some of these results have not yet been used in statistics. We also illustrate that Signflip PA performs well in numerical simulations and on empirical data examples.
