Synthetic-Powered Multiple Testing with FDR Control

2026-02-18

AI Summary

This work addresses the challenge of safely leveraging auxiliary or synthetic data to enhance statistical power while rigorously controlling the false discovery rate (FDR) in multiple hypothesis testing. The authors propose SynthBH, a novel method that, for the first time, guarantees finite-sample FDR control without requiring assumptions about the quality of synthetic data, the validity of its null p-values, or any specific distributional form. Built upon the PRDS (positive regression dependence on a subset) framework, SynthBH adaptively integrates real and synthetic data to substantially improve power while maintaining strict FDR control. Empirical evaluations demonstrate that SynthBH consistently outperforms existing approaches across diverse settings, including tabular anomaly detection, pharmacogenomic analysis of drug–cancer sensitivity associations, and simulated scenarios.

Technology Category

• Search and Optimization: Sampling/Simulation-based Search
• Natural Language Processing: Code Generation / Program Synthesis from Natural Language
• Data Mining & Knowledge Management: Anomaly/Outlier Detection

Application Category

• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

Multiple hypothesis testing with false discovery rate (FDR) control is a fundamental problem in statistical inference, with broad applications in genomics, drug screening, and outlier detection. In many such settings, researchers may have access not only to real experimental observations but also to auxiliary or synthetic data -- from past, related experiments or generated by generative models -- that can provide additional evidence about the hypotheses of interest. We introduce SynthBH, a synthetic-powered multiple testing procedure that safely leverages such synthetic data. We prove that SynthBH guarantees finite-sample, distribution-free FDR control under a mild PRDS-type positive dependence condition, without requiring the pooled-data p-values to be valid under the null. The proposed method adapts to the (unknown) quality of the synthetic data: it enhances the sample efficiency and may boost the power when synthetic data are of high quality, while controlling the FDR at a user-specified level regardless of their quality. We demonstrate the empirical performance of SynthBH on tabular outlier detection benchmarks and on genomic analyses of drug-cancer sensitivity associations, and further study its properties through controlled experiments on simulated data.

Problem

• multiple hypothesis testing
• false discovery rate
• synthetic data
• FDR control
• statistical inference

Innovation

• Synthetic data
• False Discovery Rate (FDR)
• Multiple hypothesis testing
• Distribution-free inference
• Adaptive statistical procedure