ScoreMatchingRiesz: Auto-DML with Infinitesimal Classification

2025-12-23

AI Summary

This paper addresses the susceptibility of Riesz representer estimation to overfitting in debiased machine learning. We propose an Auto-Doubly Robust Machine Learning (Auto-DML) framework grounded in score matching. Methodologically, we systematically introduce score matching into Riesz representer modeling—replacing conventional density ratio estimation—and thereby circumvent strong modeling assumptions on propensity scores or instrumental variables. By leveraging time-score functions as a bridge between marginal and average policy effects, our approach achieves √n-consistent and semiparametrically efficient estimation of causal/structural parameters. Theoretically, we establish asymptotic optimality under mild regularity conditions. Empirically, Auto-DML significantly improves estimation stability and finite-sample performance across both synthetic and real-world datasets. Moreover, it offers a novel theoretical perspective on infinitesimal classification.

Technology Category

• Machine Learning: Causal Learning
• Intelligent Robots: State Estimation
• Reasoning under Uncertainty: Relational Probabilistic Models

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: ML for personalized search and recommendations

Abstract

This study proposes Riesz representer estimation methods based on score matching. The Riesz representer is a key component in debiased machine learning for constructing $sqrt{n}$-consistent and efficient estimators in causal inference and structural parameter estimation. To estimate the Riesz representer, direct approaches have garnered attention, such as Riesz regression and the covariate balancing propensity score. These approaches can also be interpreted as variants of direct density ratio estimation (DRE) in several applications such as average treatment effect estimation. In DRE, it is well known that flexible models can easily overfit the observed data due to the estimand and the form of the loss function. To address this issue, recent work has proposed modeling the density ratio as a product of multiple intermediate density ratios and estimating it using score-matching techniques, which are often used in the diffusion model literature. We extend score-matching-based DRE methods to Riesz representer estimation. Our proposed method not only mitigates overfitting but also provides insights for causal inference by bridging marginal effects and average policy effects through time score functions.

Problem

• Estimates Riesz representer using score matching for debiased machine learning
• Addresses overfitting in direct density ratio estimation for causal inference
• Bridges marginal and average policy effects via time score functions

Innovation

• Score matching for Riesz representer estimation
• Mitigates overfitting in density ratio estimation
• Bridges marginal and average policy effects