Optimal Nuisance Function Tuning for Estimating a Doubly Robust Functional under Proportional Asymptotics

2025-09-29

AI Summary

This paper addresses the optimal tuning of ridge regression for estimating doubly robust nuisance functions—particularly the Expected Conditional Covariance (ECC)—under high-dimensional proportional asymptotics ($n,p oinfty$, $p/n ogamma>0$). Motivated by ECC’s central role in conditional independence testing and causal inference, we propose a bias-corrected ridge estimator that achieves $sqrt{n}$-consistency even without uniform estimability. We characterize the fundamental discrepancy between prediction-optimal and inference-optimal regularization parameters. Leveraging sample splitting, linear model assumptions, and asymptotic variance analysis, we construct multiple $sqrt{n}$-consistent ECC estimators, rigorously derive their asymptotic variances, and validate theoretical findings via numerical experiments. Our main contribution is a unified theoretical framework for efficient, inferentially valid ridge estimation of high-dimensional nuisance functions, enabling principled statistical inference in modern semiparametric settings.

Technology Category

• Intelligent Robots: State Estimation
• Machine Learning: Calibration & Uncertainty Quantification
• Reasoning under Uncertainty: Stochastic Optimization

Application Category

• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Security and Privacy: Large-scale security measurements

Abstract

In this paper, we explore the asymptotically optimal tuning parameter choice in ridge regression for estimating nuisance functions of a statistical functional that has recently gained prominence in conditional independence testing and causal inference. Given a sample of size $n$, we study estimators of the Expected Conditional Covariance (ECC) between variables $Y$ and $A$ given a high-dimensional covariate $X in mathbb{R}^p$. Under linear regression models for $Y$ and $A$ on $X$ and the proportional asymptotic regime $p/n o c in (0, infty)$, we evaluate three existing ECC estimators and two sample splitting strategies for estimating the required nuisance functions. Since no consistent estimator of the nuisance functions exists in the proportional asymptotic regime without imposing further structure on the problem, we first derive debiased versions of the ECC estimators that utilize the ridge regression nuisance function estimators. We show that our bias correction strategy yields $sqrt{n}$-consistent estimators of the ECC across different sample splitting strategies and estimator choices. We then derive the asymptotic variances of these debiased estimators to illustrate the nuanced interplay between the sample splitting strategy, estimator choice, and tuning parameters of the nuisance function estimators for optimally estimating the ECC. Our analysis reveals that prediction-optimal tuning parameters (i.e., those that optimally estimate the nuisance functions) may not lead to the lowest asymptotic variance of the ECC estimator -- thereby demonstrating the need to be careful in selecting tuning parameters based on the final goal of inference. Finally, we verify our theoretical results through extensive numerical experiments.

Problem

• Optimizing ridge regression tuning for doubly robust functional estimation
• Developing debiased estimators for Expected Conditional Covariance under high dimensions
• Analyzing tuning parameter selection impact on causal inference asymptotic variance

Innovation

• Uses ridge regression for nuisance function estimation
• Debiases ECC estimators with bias correction strategy
• Analyzes tuning parameter impact on asymptotic variance