Does p-Hacking Mitigate or Exacerbate the Effects of Publication Bias?

2026-09-04

AI Summary

本文研究了p-hacking对选择性发表偏差的影响，通过区分快慢p-hacking的方法，揭示其在不同选择强度下对偏差的加剧或缓解作用。

Abstract

This paper studies the effects of p-hacking on the bias of published estimates when papers with statistically significant results are selectively published. We show that fast p-hacking---actions that lead to large changes in p-values---always exacerbates the bias from selective publication. On the other hand, slow p-hacking---actions that lead to small changes in p-values---exacerbates bias when selection is weak, but mitigates it when selection is strong. In a model featuring both types of p-hacking, we show that a normality assumption identifies the true distribution of effects as well as the counterfactual mean that would obtain under selective publication without p-hacking. Applying the model to meta-analyses on the effects of behavioral nudges and development aid, we find suggestive evidence that both mitigation and exacerbation can arise in practice.

Problem

• p-hacking
• publication bias
• statistically significant results

Innovation

• p-hacking
• publication bias
• statistical significance
• normality assumption
• counterfactual mean