🤖 AI Summary
This study addresses the bias in downstream causal inference that arises when predicted variables—generated by machine learning or large language models—are used as regressors, as their inherent measurement error violates classical regression assumptions. To correct this, the authors propose a novel method that splits the original data into independent subsamples to construct multiple predicted proxies, which are then leveraged as valid instrumental variables. This approach uniquely integrates sample-splitting prediction with instrumental variable estimation, enabling unbiased causal inference without requiring external data. Simulation experiments demonstrate that the method accurately recovers true parameters even in small samples, substantially outperforming conventional techniques. Its efficacy is further corroborated in two empirical applications: one examining gendered language in the German parliament and another evaluating China’s poverty alleviation policy.
📝 Abstract
Prediction-based methods, including Large Language Models (LLMs) and other machine learning techniques, are often used to construct measures of political phenomena that are difficult to quantify directly, such as policy positions in manifestos or emotions expressed on social media. In many applications, these prediction-generated measures are used as explanatory variables in regression models, even though they are measured with error. This leads to biased estimates. In this paper, we propose a simple solution to these biases: instrumental variables constructed from multiple measures created on independent splits of the original data. This approach is theoretically valid, easy to implement, and does not require new data. Through simulations, we show that this approach recovers estimates close to the true values, even in relatively small samples, while the standard approach can produce substantial bias in practice. We illustrate the method by revisiting two applications: whether gendered speech affects legislative outcomes in the German Parliament, and whether political risk influences poverty alleviation programs in China.