Predicting Poverty

2025-05-09

AI Summary

This study investigates the robustness of counterfactual poverty rate prediction under income data missingness (MCAR/MAR). Employing synthetically generated missing-data scenarios—varying in type and proportion—we conduct the first systematic, controlled-experiment comparison of predictive accuracy and stability across linear regression, LASSO, XGBoost, and random forests. Prediction errors are decomposed against ground-truth counterfactual poverty rates to isolate bias and variance components. Results show that random forests reduce prediction error by 12–28% across most missingness settings, exhibit the smallest estimation bias, and achieve superior stability—outperforming conventional econometric models. The study establishes a reproducible evaluation framework and empirical benchmark for counterfactual inference of socioeconomic indicators under missing data, demonstrating the distinct robustness advantage of machine learning methods in policy-relevant forecasting tasks.

Technology Category

• Machine Learning: Ensemble Methods
• Reasoning under Uncertainty: Causality
• Intelligent Robots: State Estimation

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods

Abstract

Poverty prediction models are used to address missing data issues in a variety of contexts such as poverty profiling, targeting with proxy-means tests, cross-survey imputations such as poverty mapping, top and bottom incomes studies, or vulnerability analyses. Based on the models used by this literature, this paper conducts a study by artificially corrupting data clear of missing incomes with different patterns and shares of missing incomes. It then compares the capacity of classic econometric and machine learning models to predict poverty under different scenarios with full information on observed and unobserved incomes, and the true counterfactual poverty rate. Random forest provides more consistent and accurate predictions under most but not all scenarios.

Problem

• Predicting poverty with missing income data
• Comparing econometric and machine learning models
• Assessing random forest accuracy in poverty prediction

Innovation

• Uses random forest for poverty prediction
• Compares econometric and machine learning models
• Simulates missing income data patterns