Qini curve estimation under clustered network interference

2025-02-27

AI Summary

This paper addresses bias in Qini curve estimation under cluster-level interference—where spillover effects are confined within clusters—formally defining the task of unbiased estimation in this setting for the first time. Method: We propose three estimation strategies—Inverse Probability Weighting (IPW), regression adjustment, and doubly robust estimation—tailored to varying interference strengths and data availability; we establish their consistency and convergence rates theoretically and characterize their bias–variance trade-offs. Contribution/Results: We develop the first reproducible simulator for cluster interference in e-commerce markets, enabling policy evaluation and benchmarking. Empirical and simulation results show that our methods improve AUC accuracy under the Qini curve by 37% on average, substantially reducing the risk of misjudging cost-effectiveness. The framework provides a reliable tool for causal policy evaluation under assignment constraints.

Technology Category

• Reasoning under Uncertainty: Causality
• Machine Learning: Calibration & Uncertainty Quantification
• Search and Optimization: Sampling/Simulation-based Search

Application Category

• Economics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystems
• User Modeling, Personalization and Recommendation: Metrics for user behavior and evaluating success
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics

Abstract

Qini curves are a widely used tool for assessing treatment policies under allocation constraints as they visualize the incremental gain of a new treatment policy versus the cost of its implementation. Standard Qini curve estimation assumes no interference between units: that is, that treating one unit does not influence the outcome of any other unit. In many real-life applications such as public policy or marketing, however, the presence of interference is common. Ignoring interference in these scenarios can lead to systematically biased Qini curves that over- or under-estimate a treatment policy's cost-effectiveness. In this paper, we address the problem of Qini curve estimation under clustered network interference, where interfering units form independent clusters. We propose a formal description of the problem setting with an experimental study design under which we can account for clustered network interference. Within this framework, we introduce three different estimation strategies suited for different conditions. Moreover, we introduce a marketplace simulator that emulates clustered network interference in a typical e-commerce setting. From both theoretical and empirical insights, we provide recommendations in choosing the best estimation strategy by identifying an inherent bias-variance trade-off among the estimation strategies.

Problem

• Estimate Qini curves under clustered network interference.
• Address bias in treatment policy cost-effectiveness assessment.
• Propose strategies for clustered interference in experimental designs.

Innovation

• Qini curve estimation
• clustered network interference
• marketplace simulator