Estimating the True Effect Size Distribution with SIMEX

2026-08-21

AI Summary

本文提出一种基于SIMEX的非参数方法，用于从带有已知方差的估计平均处理效应中估计真实效应大小的潜在分布，以解决大规模在线实验产生的噪声效应估计问题。

Abstract

Large-scale online experimentation produces noisy effect estimates, which can overstate gains and complicate decisions about launches and testing policies. We propose a nonparametric method based on SIMulation-EXtrapolation (SIMEX) to estimate the latent distribution of true effects from estimated average treatment effects with known variances. The method evaluates quantiles after adding progressively more simulated measurement noise and extrapolates the resulting inverse cumulative distribution function to the zero-noise setting while enforcing monotonicity of the quantiles. In a synthetic example with normally distributed true effects and measurement error, the method recovers the underlying effect distribution and performs nearly as well as a parametric empirical Bayes normal means approach. This provides a flexible way to characterize effect-size distributions without fully specifying a parametric model.

Problem

• Large-scale online experimentation
• noisy effect estimates
• latent distribution of true effects
• SIMulation-EXtrapolation (SIMEX)

Innovation

• SIMEX
• nonparametric method
• latent distribution
• measurement noise
• extrapolation