Denoised IPW-Lasso for Heterogeneous Treatment Effect Estimation in Randomized Experiments

2025-10-12

AI Summary

In randomized experiments, estimation of the conditional average treatment effect (CATE) suffers from high variance and low accuracy when treatment effects are weak. To address this, we propose DeNoised IPW-Lasso: a method that integrates inverse probability weighting (IPW) with Lasso regularization and incorporates a theory-driven denoising strategy to reduce prediction error. We theoretically establish that this denoising mechanism substantially improves the estimation efficiency of Lasso under small treatment effects. The resulting estimator is both highly interpretable and robust to model misspecification. Evaluated on two real-world datasets—voter mobilization and online advertising uplift—the method outperforms state-of-the-art nonparametric machine learning approaches. It accurately recovers heterogeneous treatment patterns consistent with empirical domain knowledge. Thus, DeNoised IPW-Lasso provides an efficient, interpretable, and principled tool for causal heterogeneity analysis.

Technology Category

• Machine Learning: Causal Learning
• Reasoning under Uncertainty: Causality
• Intelligent Robots: State Estimation

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

This paper proposes a new method for estimating conditional average treatment effects (CATE) in randomized experiments. We adopt inverse probability weighting (IPW) for identification; however, IPW-transformed outcomes are known to be noisy, even when true propensity scores are used. To address this issue, we introduce a noise reduction procedure and estimate a linear CATE model using Lasso, achieving both accuracy and interpretability. We theoretically show that denoising reduces the prediction error of the Lasso. The method is particularly effective when treatment effects are small relative to the variability of outcomes, which is often the case in empirical applications. Applications to the Get-Out-the-Vote dataset and Criteo Uplift Modeling dataset demonstrate that our method outperforms fully nonparametric machine learning methods in identifying individuals with higher treatment effects. Moreover, our method uncovers informative heterogeneity patterns that are consistent with previous empirical findings.

Problem

• Estimating conditional average treatment effects in experiments
• Reducing noise in inverse probability weighting outcomes
• Improving accuracy and interpretability of CATE estimation

Innovation

• Denoised IPW reduces outcome noise
• Lasso estimates linear CATE model
• Combines accuracy with interpretability