The purpose of an estimator is what it does: Misspecification, estimands, and over-identification

2025-08-18

AI Summary

In overidentified models, misspecification is pervasive, causing distinct estimators to target fundamentally different parameters—not merely differing in efficiency. This paper, grounded in the Generalized Method of Moments (GMM) framework, demonstrates how the choice of weighting matrix systematically alters the estimand. It further establishes that Hansen’s J-statistic asymptotically characterizes the feasible range of point estimates under a given standard error, thereby quantifying the “estimation degrees of freedom” problem. The analysis reveals that commonly used inefficient estimators implicitly amplify researcher discretion, undermining result comparability and transparency. Our main contributions are threefold: (1) clarifying the non-equivalence between estimands and estimators under misspecification; (2) reinterpreting the J-statistic as a measure of estimand uncertainty; and (3) advocating for the routine reporting of the J-statistic to enhance empirical robustness and reproducibility.

Technology Category

• Reasoning under Uncertainty: Relational Probabilistic Models
• Machine Learning: Calibration & Uncertainty Quantification
• Intelligent Robots: State Estimation

Application Category

• User Modeling, Personalization and Recommendation: Metrics for user behavior and evaluating success
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods
• Security and Privacy: Large-scale security measurements

Abstract

In over-identified models, misspecification -- the norm rather than exception -- fundamentally changes what estimators estimate. Different estimators imply different estimands rather than different efficiency for the same target. A review of recent applications of generalized method of moments in the American Economic Review suggests widespread acceptance of this fact: There is little formal specification testing and widespread use of estimators that would be inefficient were the model correct, including the use of "hand-selected" moments and weighting matrices. Motivated by these observations, we review and synthesize recent results on estimation under model misspecification, providing guidelines for transparent and robust empirical research. We also provide a new theoretical result, showing that Hansen's J-statistic measures, asymptotically, the range of estimates achievable at a given standard error. Given the widespread use of inefficient estimators and the resulting researcher degrees of freedom, we thus particularly recommend the broader reporting of J-statistics.

Problem

• Examining how misspecification alters estimands in over-identified models
• Analyzing inefficient estimator use in generalized method of moments
• Proposing J-statistics reporting for robust empirical research

Innovation

• Generalized method of moments analysis
• Hansen's J-statistic measures estimates
• Guidelines for robust empirical research