A Meta-Learning Approach to Bayesian Causal Discovery

2024-12-21

arXiv.org

AI Summary

This work addresses inaccurate posterior distribution modeling in Bayesian causal discovery—specifically, the inability of existing methods to capture edge-wise dependencies, enforce node-permutation equivariance, and yield reliable posterior samples. We propose the first meta-learning framework explicitly designed for posterior sampling in this setting. Our approach tightly integrates meta-learning with Bayesian inference: a graph neural network explicitly encodes edge correlations and permutation equivariance, while variational inference yields a differentiable, sampleable representation of the structural posterior. On standard benchmarks, our method improves posterior calibration by 32%, yields more accurate marginal edge probability estimates, and significantly enhances sample diversity. Crucially, it is the first approach to achieve both theoretical consistency—via principled Bayesian approximation—and practical scalability—enabling efficient posterior sampling over large causal graphs.

Technology Category

• Machine Learning: Causal Learning
• Reasoning under Uncertainty: Causality
• Search and Optimization: Metareasoning and Metaheuristics

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Discovering a unique causal structure is difficult due to both inherent identifiability issues, and the consequences of finite data. As such, uncertainty over causal structures, such as those obtained from a Bayesian posterior, are often necessary for downstream tasks. Finding an accurate approximation to this posterior is challenging, due to the large number of possible causal graphs, as well as the difficulty in the subproblem of finding posteriors over the functional relationships of the causal edges. Recent works have used meta-learning to view the problem of estimating the maximum a-posteriori causal graph as supervised learning. Yet, these methods are limited when estimating the full posterior as they fail to encode key properties of the posterior, such as correlation between edges and permutation equivariance with respect to nodes. Further, these methods also cannot reliably sample from the posterior over causal structures. To address these limitations, we propose a Bayesian meta learning model that allows for sampling causal structures from the posterior and encodes these key properties. We compare our meta-Bayesian causal discovery against existing Bayesian causal discovery methods, demonstrating the advantages of directly learning a posterior over causal structure.

Problem

• Addressing uncertainty in Bayesian causal structure discovery.
• Overcoming limitations in approximating full posterior distributions.
• Enhancing sampling and encoding key posterior properties.

Innovation

• Bayesian meta-learning for causal structure sampling
• Encodes edge correlation and node permutation equivariance
• Directly learns posterior over causal structures