Bayesian Additive Regression Tree Copula Processes for Scalable Distributional Prediction

2026-01-08

AI Summary

This work addresses the limitations of traditional Bayesian Additive Regression Trees (BART) in flexibly modeling the full conditional distribution of response variables and its restricted scalability. The authors propose a novel approach that introduces a prior on the leaf-node variances of BART to construct an implicit copula process, which is then coupled with arbitrary marginal distributions—marking the first integration of copula processes with BART. By leveraging optimal transport maps, the method yields a closed-form posterior predictive distribution, substantially improving distributional prediction accuracy while preserving scalability in high dimensions. Extensive experiments on simulated data and five real-world datasets (with sample sizes ranging from 506 to 515,345) demonstrate that the proposed method consistently outperforms standard BART and leading benchmarks in terms of accuracy, flexibility, and scalability.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Reasoning under Uncertainty: Relational Probabilistic Models
• Data Mining & Knowledge Management: Scalability, Parallel & Distributed Systems

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

We show how to construct the implied copula process of response values from a Bayesian additive regression tree (BART) model with prior on the leaf node variances. This copula process, defined on the covariate space, can be paired with any marginal distribution for the dependent variable to construct a flexible distributional BART model. Bayesian inference is performed via Markov chain Monte Carlo on an augmented posterior, where we show that key sampling steps can be realized as those of Chipman et al. (2010), preserving scalability and computational efficiency even though the copula process is high dimensional. The posterior predictive distribution from the copula process model is derived in closed form as the push-forward of the posterior predictive distribution of the underlying BART model with an optimal transport map. Under suitable conditions, we establish posterior consistency for the regression function and posterior means and prove convergence in distribution of the predictive process and conditional expectation. Simulation studies demonstrate improved accuracy of distributional predictions compared to the original BART model and leading benchmarks. Applications to five real datasets with 506 to 515,345 observations and 8 to 90 covariates further highlight the efficacy and scalability of our proposed BART copula process model.

Problem

• distributional prediction
• Bayesian additive regression trees
• copula process
• scalability
• posterior consistency

Innovation

• Bayesian Additive Regression Trees
• Copula Process
• Distributional Prediction
• Optimal Transport
• Scalable Inference