Correction for nonignorable nonresponse bias in the estimation of turnout using callback data

2025-04-19

AI Summary

In election surveys, nonignorable nonresponse bias frequently leads to upward bias in turnout estimates. This paper proposes a causal calibration framework leveraging callback records and census covariates, establishing identifiability of the turnout parameter under nonmonotone missingness not at random (NMAR) for the first time—via a novel “robust stability” assumption. The method integrates callback-pattern modeling, covariate-assisted identification, doubly robust estimation, and census-level auxiliary information. Empirical results demonstrate that the estimator closely approximates official turnout figures and accurately captures the declining trend in voting propensity as contact difficulty increases. The core contributions are threefold: (i) it overcomes the theoretical identifiability barrier for turnout estimation under NMAR; (ii) it yields the first callback-based correction method that is simultaneously identifiable, robust to model misspecification, and practically implementable; and (iii) it provides principled integration of administrative and survey data for improved inference.

Technology Category

• Reasoning under Uncertainty: Causality
• Machine Learning: Calibration & Uncertainty Quantification
• Intelligent Robots: State Estimation

Application Category

• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Economics, Online Markets and Human Computation: Trust and reliance of crowd workers and data experts on GenAI
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Overestimation of turnout has long been an issue in election surveys, with nonresponse bias or voter overrepresentation regarded as one of the major sources of bias. However, the adjustment for nonignorable nonresponse bias is substantially challenging. Based on the ANES Non-Response Follow-Up Study concerning the 2020 U.S. presidential election, we investigate the role of callback data in adjusting for nonresponse bias in the estimation of turnout. Callback data are the records of contact attempts in the survey course, available in many modern large-scale surveys. We propose a stableness of resistance assumption to account for the nonignorable missingness in the outcome, which states that the impact of the missing outcome on the response propensity is stable in the first two call attempts. Under this assumption and by leveraging covariates information from the census data, we establish the identifiability and develop estimation methods for turnout, including a doubly robust estimator. Our methods produce estimates very close to the official turnout and successfully capture the trend of declining willingness to vote as response reluctance or contact difficulty increases. This work hints at the importance of adjusting for nonignorable nonresponse bias and exhibits the promise of callback data for political surveys.

Problem

• Adjusting nonignorable nonresponse bias in turnout estimation
• Overcoming overestimation of turnout in election surveys
• Utilizing callback data to correct nonresponse bias

Innovation

• Uses callback data for nonresponse bias adjustment
• Proposes stableness of resistance assumption
• Develops doubly robust estimator for turnout