Ultrahigh-dimensional Quadratic Discriminant Analysis Using Random Projections

2025-05-29

AI Summary

In ultra-high-dimensional, small-sample classification (where (p gg n)), classical Quadratic Discriminant Analysis (QDA) fails due to the singularity of the sample covariance matrix. To address this, we propose RPE-QDA—a novel integration of Random Projection Ensemble (RPE) with QDA, accompanied by the first theoretical guarantees for such an approach. Under mild conditions—namely, sub-exponential growth of (p) with (n) and sub-exponential tails of features—we establish that RPE-QDA achieves asymptotically perfect classification, attaining both statistical optimality and computational feasibility. Key technical innovations include covariance estimation via the Moore–Penrose pseudoinverse and rigorous support from high-dimensional random matrix theory. Extensive experiments on synthetic data and real gene expression datasets demonstrate that RPE-QDA consistently outperforms baseline methods—including LDA, SPAM, and ROAD—with average accuracy improvements of 12%–35% and two orders-of-magnitude reduction in computational time.

Technology Category

• Machine Learning: Dimensionality Reduction/Feature Selection
• Search and Optimization: Mixed Discrete/Continuous Search
• Reasoning under Uncertainty: Stochastic Optimization

Application Category

• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets
• Search and Retrieval-Augmented AI: Efficiency and scalability of Web search engines
• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs

Abstract

This paper investigates the effectiveness of implementing the Random Projection Ensemble (RPE) approach in Quadratic Discriminant Analysis (QDA) for ultrahigh-dimensional classification problems. Classical methods such as Linear Discriminant Analysis (LDA) and QDA are widely used, but face significant challenges when the dimension of the samples (say, $p$) exceeds the sample size (say, $n$). In particular, both LDA (using the Moore-Penrose inverse for covariance matrices) and QDA (even with known covariance matrices) may perform as poorly as random guessing when $p/n o infty$. The RPE method, known for addressing the curse of dimensionality, offers a fast and effective solution. This paper demonstrates the practical advantages of employing RPE on QDA in terms of classification performance as well as computational efficiency. We establish results for limiting perfect classification in both the population and sample versions of the proposed RPE-QDA classifier, under fairly general assumptions that allow for sub-exponential growth of $p$ relative to $n$. Several simulated and gene expression datasets are used to evaluate the performance of the proposed classifier in ultrahigh-dimensional scenarios.

Problem

• Address ultrahigh-dimensional classification challenges in QDA
• Overcome poor performance when dimension exceeds sample size
• Enhance QDA with Random Projection Ensemble for efficiency

Innovation

• Random Projection Ensemble for QDA
• Handles ultrahigh-dimensional classification
• Ensures computational efficiency