An Efficient Framework for Robust Sample Size Determination

2025-12-18

AI Summary

Traditional sample size determination methods rely on a single assumed data-generating mechanism, require extensive simulations, and exhibit poor generalizability. To address these limitations, this paper proposes a robust sample size design framework. The core methodological innovation is the first derivation of an analytical functional model of p-values as a function of sample size, integrated with M-estimation, asymptotic p-value modeling, and cross-mechanism sensitivity analysis—enabling full-power evaluation across the entire sample space using only two simulations per data-generating mechanism. This approach relaxes stringent assumptions about the underlying generative process, substantially enhancing robustness and computational efficiency. Empirical validation across multiple clinical trials and observational studies demonstrates that the proposed method reduces sample size recommendation computation time by over 90% while maintaining statistical power estimation error within 2%.

Technology Category

• Search and Optimization: Sampling/Simulation-based Search
• Machine Learning: Calibration & Uncertainty Quantification
• Multiagent Systems: Mechanism Design

Application Category

• User Modeling, Personalization and Recommendation: Practical large-scale studies of user experience
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods
• Security and Privacy: Large-scale security measurements

Abstract

In many settings, robust data analysis involves computational methods for uncertainty quantification and statistical inference. To design frequentist studies that leverage robust analysis methods, suitable sample sizes to achieve desired power are often found by estimating sampling distributions of p-values via intensive simulation. Moreover, most sample size recommendations rely heavily on assumptions about a single data-generating process. Consequently, robustness in data analysis does not by itself imply robustness in study design, as examining sample size sensitivity to data-generating assumptions typically requires further simulations. We propose an economical alternative for determining sample sizes that are robust to multiple data-generating mechanisms. Applying our theoretical results that model p-values as a function of the sample size, we assess power across the sample size space using simulations conducted at only two sample sizes for each data-generating mechanism. We demonstrate the broad applicability of our methodology to study design based on M-estimators in both experimental and observational settings through a varied set of clinical examples.

Problem

• Determines robust sample sizes for multiple data-generating mechanisms
• Assesses power efficiently using simulations at only two sample sizes
• Applies to study design with M-estimators in clinical settings

Innovation

• Models p-values as sample size function
• Assesses power using two-sample simulations
• Applies to M-estimators in clinical studies