Bayesian Hierarchical Invariant Prediction

2025-05-16

AI Summary

This paper addresses the computational intractability and difficulty of incorporating prior knowledge in causal mechanism invariance testing under heterogeneous data in high-dimensional settings. We propose the Bayesian Hierarchical Invariant Prediction (BHIP) framework, which reformulates Invariant Causal Prediction (ICP) as a hierarchical Bayesian model for the first time. BHIP employs a joint horseshoe and spike-and-slab sparse prior, substantially enhancing statistical reliability and robustness in causal feature identification, while enabling flexible integration of domain knowledge via interpretable priors. Theoretically grounded and empirically validated, BHIP demonstrates superior statistical power, stronger robustness to distributional shifts, and faster computation than ICP on both synthetic and real-world datasets. By unifying invariance-based causal discovery with scalable Bayesian inference, BHIP provides a computationally tractable, statistically principled, and interpretable solution for high-dimensional heterogeneous environments.

Technology Category

• Machine Learning: Causal Learning
• Reasoning under Uncertainty: Causality
• Knowledge Representation and Reasoning: Action, Change, and Causality

Application Category

• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs
• Economics, Online Markets and Human Computation: Fairness, privacy, and diversity in economic environments
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems

Abstract

We propose Bayesian Hierarchical Invariant Prediction (BHIP) reframing Invariant Causal Prediction (ICP) through the lens of Hierarchical Bayes. We leverage the hierarchical structure to explicitly test invariance of causal mechanisms under heterogeneous data, resulting in improved computational scalability for a larger number of predictors compared to ICP. Moreover, given its Bayesian nature BHIP enables the use of prior information. In this paper, we test two sparsity inducing priors: horseshoe and spike-and-slab, both of which allow us a more reliable identification of causal features. We test BHIP in synthetic and real-world data showing its potential as an alternative inference method to ICP.

Problem

• Proposes Bayesian Hierarchical Invariant Prediction for causal inference
• Improves scalability and reliability in identifying causal features
• Tests sparsity-inducing priors on synthetic and real-world data

Innovation

• Bayesian Hierarchical Invariant Prediction reframes ICP
• Leverages hierarchical structure for invariance testing
• Uses sparsity-inducing priors for causal feature identification