False discovery rate control with compound p-values

2025-07-28

AI Summary

This paper investigates the false discovery rate (FDR) control performance of the Benjamini–Hochberg (BH) procedure under *composite p-values*—i.e., p-values that are only required to be super-uniform on all true null hypotheses, a weaker condition than standard uniformity. Theoretical analysis establishes that, under independence, FDR ≤ 1.93α; when all nulls are true, FDR ≤ α + 2α²; and under positive dependence, FDR may inflate by a factor of O(log m), with matching tight upper and lower bound constructions. This work provides the first systematic characterization of the robustness boundary of the BH procedure to composite p-values, precisely delineating its FDR control capability—and inherent limitations—when the classical uniformity assumption is relaxed. The results establish a new theoretical foundation for multiple testing in high-dimensional settings with complex dependencies, enhancing both flexibility and statistical power.

Technology Category

• Reasoning under Uncertainty: Other Foundations of Reasoning under Uncertainty
• Search and Optimization: Mixed Discrete/Continuous Search
• Machine Learning: Calibration & Uncertainty Quantification

Application Category

• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Security and Privacy: Large-scale security measurements

Abstract

In the setting of multiple testing, compound p-values generalize p-values by asking for superuniformity to hold only emph{on average} across all true nulls. We study the properties of the Benjamini--Hochberg procedure applied to compound p-values. Under independence, we show that the false discovery rate (FDR) is at most $1.93α$, where $α$ is the nominal level, and exhibit a distribution for which the FDR is $frac{7}{6}α$. If additionally all nulls are true, then the upper bound can be improved to $α+ 2α^2$, with a corresponding worst-case lower bound of $α+ α^2/4$. Under positive dependence, on the other hand, we demonstrate that FDR can be inflated by a factor of $O(log m)$, where~$m$ is the number of hypotheses. We provide numerous examples of settings where compound p-values arise in practice, either because we lack sufficient information to compute non-trivial p-values, or to facilitate a more powerful analysis.

Problem

• Study FDR control using compound p-values in multiple testing
• Analyze Benjamini-Hochberg procedure's performance with compound p-values
• Explore FDR bounds under independence and positive dependence

Innovation

• Uses compound p-values for FDR control
• Applies Benjamini-Hochberg procedure to compound p-values
• Analyzes FDR bounds under independence and dependence