Statistical Inference under Performativity

2025-05-24

AI Summary

This paper addresses the statistical inference challenge under *predictive executability*—where prediction outcomes retroactively influence the predicted target—rendering conventional causal and predictive inference assumptions invalid and leading to miscalibrated confidence intervals and biased hypothesis tests in policy evaluation. To resolve this, we establish the first systematic theoretical framework for executability-aware inference: (i) we derive a central limit theorem tailored to predictive executability, ensuring asymptotic normality of estimators; and (ii) we propose Predictive-Enhanced Inference (PPI), a novel method that jointly leverages small-scale labeled data and large-scale model predictions to achieve unbiased, efficient estimation of policy parameters. We provide rigorous theoretical guarantees showing that PPI substantially improves estimation accuracy and confidence region coverage. Extensive numerical experiments demonstrate its strong robustness and practical efficacy in real-world policy evaluation tasks.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Reasoning under Uncertainty: Probabilistic Inference
• Humans and AI: Human-Aware Planning and Behavior Prediction

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Security and Privacy: Large-scale security measurements

Abstract

Performativity of predictions refers to the phenomena that prediction-informed decisions may influence the target they aim to predict, which is widely observed in policy-making in social sciences and economics. In this paper, we initiate the study of statistical inference under performativity. Our contribution is two-fold. First, we build a central limit theorem for estimation and inference under performativity, which enables inferential purposes in policy-making such as constructing confidence intervals or testing hypotheses. Second, we further leverage the derived central limit theorem to investigate prediction-powered inference (PPI) under performativity, which is based on a small labeled dataset and a much larger dataset of machine-learning predictions. This enables us to obtain more precise estimation and improved confidence regions for the model parameter (i.e., policy) of interest in performative prediction. We demonstrate the power of our framework by numerical experiments. To the best of our knowledge, this paper is the first one to establish statistical inference under performativity, which brings up new challenges and inference settings that we believe will add significant values to policy-making, statistics, and machine learning.

Problem

• Studying statistical inference under performativity in predictions
• Establishing central limit theorem for performative policy-making inference
• Enhancing prediction-powered inference with performative prediction models

Innovation

• Central limit theorem for performative predictions
• Prediction-powered inference with large datasets
• Improved confidence regions for policy parameters