Bayesian Double Machine Learning for Causal Inference

2025-08-18

AI Summary

In high-dimensional settings with partial linear models, machine learning regularization induces confounding bias in causal inference. To address this, we propose the first fully Bayesian double machine learning (DML) framework. Our method modifies the Bayesian multivariate regression model to identify causal effects from the reduced-form covariance matrix, seamlessly integrating Bayesian modeling with DML to construct a generative probabilistic model that strictly adheres to the likelihood principle—without requiring selective observability assumptions. We establish theoretical guarantees: asymptotic unbiasedness, normality, and semiparametric efficiency under finite samples, justified by the Bernstein–von Mises theorem. Simulation studies demonstrate that our approach significantly reduces mean squared error compared to existing Bayesian and frequentist methods, while improving confidence interval coverage probability and narrowing interval width. It further exhibits superior robustness and statistical efficiency.

Technology Category

• Machine Learning: Bayesian Learning
• Reasoning under Uncertainty: Causality
• Cognitive Modeling & Cognitive Systems: Conceptual Inference and Reasoning

Application Category

• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

This paper proposes a simple, novel, and fully-Bayesian approach for causal inference in partially linear models with high-dimensional control variables. Off-the-shelf machine learning methods can introduce biases in the causal parameter known as regularization-induced confounding. To address this, we propose a Bayesian Double Machine Learning (BDML) method, which modifies a standard Bayesian multivariate regression model and recovers the causal effect of interest from the reduced-form covariance matrix. Our BDML is related to the burgeoning frequentist literature on DML while addressing its limitations in finite-sample inference. Moreover, the BDML is based on a fully generative probability model in the DML context, adhering to the likelihood principle. We show that in high dimensional setups the naive estimator implicitly assumes no selection on observables--unlike our BDML. The BDML exhibits lower asymptotic bias and achieves asymptotic normality and semiparametric efficiency as established by a Bernstein-von Mises theorem, thereby ensuring robustness to misspecification. In simulations, our BDML achieves lower RMSE, better frequentist coverage, and shorter confidence interval width than alternatives from the literature, both Bayesian and frequentist.

Problem

• Addresses regularization-induced confounding in causal inference
• Proposes Bayesian Double Machine Learning for high-dimensional models
• Ensures robustness and efficiency in finite-sample inference

Innovation

• Bayesian Double Machine Learning method
• Generative probability model adherence
• Reduced asymptotic bias robustness