The Effect of Omitted Variables on the Sign of Regression Coefficients

2022-08-01

AI Summary

Existing robustness metrics—such as Oster’s δ—overestimate causal inference reliability by relying on the “coefficient-to-zero” assumption, thereby severely underestimating how omitted variables destabilize sign consistency of regression coefficients; in practice, omitted variables more readily induce sign reversal than mere attenuation toward zero. Method: We propose a revised sensitivity framework for sign stability, introducing novel robustness metrics grounded in the critical threshold for sign reversal rather than coefficient nullification. Our approach integrates causal inference principles, structured sensitivity analysis, and regression diagnostics, and we implement it via the Stata command `regsensitivity` for automated assessment. Contribution/Results: Applied to four empirical studies and two meta-analyses, our metrics substantially improve detection of sign misclassification risk. They provide a more credible robustness benchmark for causal inference, directly addressing the limitations of conventional sensitivity measures predicated on coefficient shrinkage.

Abstract

We show that, depending on how the impact of omitted variables is measured, it can be substantially easier for omitted variables to flip coefficient signs than to drive them to zero. This behavior occurs with"Oster's delta"(Oster 2019), a widely reported robustness measure. Consequently, any time this measure is large -- suggesting that omitted variables may be unimportant -- a much smaller value reverses the sign of the parameter of interest. We propose a modified measure of robustness to address this concern. We illustrate our results in four empirical applications and two meta-analyses. We implement our methods in the companion Stata module regsensitivity.

Problem

• Omitted variables can flip regression coefficient signs
• Oster's delta may misleadingly suggest robustness
• Proposed modified measure addresses sign reversal risk

Innovation

• Modified robustness measure addresses sign reversal
• Uses Oster's delta for omitted variables impact
• Implemented in Stata module regsensitivity