Double Machine Learning for Conditional Moment Restrictions: IV regression, Proximal Causal Learning and Beyond

2025-06-17

AI Summary

This paper addresses unbiased function estimation under conditional moment restrictions (CMRs), focusing on key causal inference tasks such as instrumental variable (IV) regression and proximal causal learning. Existing two-stage estimators suffer from substantial bias due to plug-in substitution of first-stage predictions—particularly severe when deep neural networks are employed, owing to regularization effects and overfitting. To overcome this, we propose DML-CMR: a debiased machine learning estimator that integrates dual-stage deep networks, residualization-based debiasing, moment condition optimization, and nonparametric statistical theory. DML-CMR is the first estimator to achieve the minimax-optimal $O(N^{-1/2})$ convergence rate under CMRs. Experiments on real-world data demonstrate state-of-the-art performance in both IV regression and proximal causal learning, significantly outperforming existing CMR-based and task-specific causal inference methods.

Technology Category

• Machine Learning: Causal Learning
• Reasoning under Uncertainty: Causality
• Intelligent Robots: State Estimation

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Web Mining and Content Analysis: Machine learning and data science for the Web
• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs

Abstract

Solving conditional moment restrictions (CMRs) is a key problem considered in statistics, causal inference, and econometrics, where the aim is to solve for a function of interest that satisfies some conditional moment equalities. Specifically, many techniques for causal inference, such as instrumental variable (IV) regression and proximal causal learning (PCL), are CMR problems. Most CMR estimators use a two-stage approach, where the first-stage estimation is directly plugged into the second stage to estimate the function of interest. However, naively plugging in the first-stage estimator can cause heavy bias in the second stage. This is particularly the case for recently proposed CMR estimators that use deep neural network (DNN) estimators for both stages, where regularisation and overfitting bias is present. We propose DML-CMR, a two-stage CMR estimator that provides an unbiased estimate with fast convergence rate guarantees. We derive a novel learning objective to reduce bias and develop the DML-CMR algorithm following the double/debiased machine learning (DML) framework. We show that our DML-CMR estimator can achieve the minimax optimal convergence rate of $O(N^{-1/2})$ under parameterisation and mild regularity conditions, where $N$ is the sample size. We apply DML-CMR to a range of problems using DNN estimators, including IV regression and proximal causal learning on real-world datasets, demonstrating state-of-the-art performance against existing CMR estimators and algorithms tailored to those problems.

Problem

• Reducing bias in two-stage CMR estimators with DNNs
• Improving convergence rates for IV regression and PCL
• Developing unbiased DML-CMR algorithm for optimal performance

Innovation

• Uses double/debiased machine learning framework
• Reduces bias with novel learning objective
• Achieves minimax optimal convergence rate