Perturbed Double Machine Learning: Nonstandard Inference Beyond the Parametric Length

2025-11-02

AI Summary

This paper addresses robust inference for a low-dimensional target parameter β in the presence of infinite-dimensional nuisance parameters. Conventional double machine learning (DML) requires nuisance estimators to converge at a rate faster than n⁻¹/⁴; otherwise, Wald-type confidence intervals become invalid. To overcome this limitation, we propose Perturbed Double Machine Learning (P-DML): it injects random perturbations into nuisance estimation to generate multiple β estimates, then applies bias screening and thresholding to retain only valid ones—thereby relaxing the n⁻¹/⁴ rate requirement. P-DML accommodates both Lasso and general machine learning algorithms without imposing strong convergence conditions on nuisance estimators. We establish that the resulting estimator possesses oracle properties and yields asymptotically valid confidence intervals. Simulation studies demonstrate that P-DML maintains nominal coverage even when nuisance parameters converge slowly, substantially enhancing the robustness of statistical inference.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Intelligent Robots: State Estimation
• Reasoning under Uncertainty: Probabilistic Inference

Application Category

• User Modeling, Personalization and Recommendation: ML for personalized search and recommendations
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

We study inference on a low dimensional functional $β$ in the presence of possibly infinite dimensional nuisance parameters. Classical inferential methods are typically based on the Wald interval, whose large sample validity rests on the asymptotic negligibility of the nuisance error; for example, estimators based on the influence curve of the parameter (Double/Debiased Machine Learning DML estimators) are asymptotically Gaussian when the nuisance estimators converge at rates faster than $n^{-1/4}$. Although, under suitable conditions, such negligibility can hold even in nonparametric classes, it can be restrictive. To relax this requirement, we propose Perturbed Double Machine Learning (Perturbed DML) to ensure valid inference even when nuisance estimators converge at rates slower than $n^{-1/4}$. Our proposal is to 1) inject randomness into the nuisance estimation step to generate a collection of perturbed nuisance models, each yielding an estimate of $β$ and a corresponding Wald interval, and 2) filter out perturbations whose deviations from the original DML estimate exceed a threshold. For Lasso nuisance learners, we show that, with high probability, at least one perturbation produces nuisance estimates sufficiently close to the truth, so that the associated estimator of $β$ is close to an oracle estimator with knowledge of the true nuisances. Taking the union of the retained intervals delivers valid coverage even when the DML estimator converges more slowly than $n^{-1/2}$. The framework extends to general machine learning nuisance learners, and simulations show that Perturbed DML can have coverage when state of the art methods fail.

Problem

• Inference on low-dimensional parameters with infinite-dimensional nuisances
• Valid inference when nuisance estimators converge slower than n^{-1/4}
• Robust coverage for DML estimators using perturbation and filtering

Innovation

• Inject randomness into nuisance estimation for robustness
• Filter perturbations based on deviation from original estimate
• Union of retained intervals ensures valid statistical coverage