Causal Bayesian Optimization via Exogenous Distribution Learning

2024-02-03

arXiv.org

AI Summary

This work addresses the problem of optimizing target variables in structural causal models (SCMs), particularly under small-sample and high-uncertainty regimes, where conventional additive noise models (ANMs) impose restrictive assumptions. We propose a novel causal Bayesian optimization framework that explicitly models and learns the distribution of exogenous variables—enabling flexible, data-driven priors over noise and latent confounders—and thereby improves surrogate model fidelity to underlying causal mechanisms without requiring hard interventions. Our method jointly integrates exogenous distribution learning, SCM-based causal modeling, and observation-driven Bayesian optimization. Extensive experiments on synthetic benchmarks and real-world datasets demonstrate significant improvements in both optimization performance and robustness, especially with limited data. Moreover, our approach generalizes causal optimization beyond ANM-restricted SCMs, substantially broadening its applicability to general non-additive causal structures.

Technology Category

• Machine Learning: Causal Learning
• Reasoning under Uncertainty: Stochastic Optimization
• Search and Optimization: Sampling/Simulation-based Search

Application Category

• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems

Abstract

Maximizing a target variable as an operational objective in a structural causal model is an important problem. Existing Causal Bayesian Optimization~(CBO) methods either rely on hard interventions that alter the causal structure to maximize the reward; or introduce action nodes to endogenous variables so that the data generation mechanisms are adjusted to achieve the objective. In this paper, a novel method is introduced to learn the distribution of exogenous variables, which is typically ignored or marginalized through expectation by existing methods. Exogenous distribution learning improves the approximation accuracy of structural causal models in a surrogate model that is usually trained with limited observational data. Moreover, the learned exogenous distribution extends existing CBO to general causal schemes beyond Additive Noise Models~(ANM). The recovery of exogenous variables allows us to use a more flexible prior for noise or unobserved hidden variables. We develop a new CBO method by leveraging the learned exogenous distribution. Experiments on different datasets and applications show the benefits of our proposed method.

Problem

• Causal Inference
• Optimization under Uncertainty
• Additive Noise Models

Innovation

• Bayesian Approach
• Causal Inference Optimization
• External Data Distribution Learning