Bayesian Additive Regression Trees for functional ANOVA model

2025-09-03

AI Summary

While BART offers strong predictive performance, it suffers from poor interpretability due to its black-box nature. Method: We propose ANOVA-BART—the first integration of functional ANOVA decomposition into the BART framework—explicitly modeling main effects and interactions of all orders while preserving Bayesian coherence and high predictive accuracy. Our method achieves near minimax-optimal posterior contraction rates and provides individual convergence guarantees for each interaction term. Leveraging MCMC sampling coupled with an interaction selection algorithm, it enables identifiable and interpretable component screening. Results: Experiments demonstrate that ANOVA-BART significantly outperforms standard BART in prediction accuracy, uncertainty quantification, and interpretability. It establishes a new paradigm for complex nonlinear modeling that simultaneously ensures statistical rigor and transparency.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Reasoning under Uncertainty: Graphical Models
• Natural Language Processing: Interpretability, Analysis, and Evaluation of NLP Models

Application Category

• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Bayesian Additive Regression Trees (BART) is a powerful statistical model that leverages the strengths of Bayesian inference and regression trees. It has received significant attention for capturing complex non-linear relationships and interactions among predictors. However, the accuracy of BART often comes at the cost of interpretability. To address this limitation, we propose ANOVA Bayesian Additive Regression Trees (ANOVA-BART), a novel extension of BART based on the functional ANOVA decomposition, which is used to decompose the variability of a function into different interactions, each representing the contribution of a different set of covariates or factors. Our proposed ANOVA-BART enhances interpretability, preserves and extends the theoretical guarantees of BART, and achieves superior predictive performance. Specifically, we establish that the posterior concentration rate of ANOVA-BART is nearly minimax optimal, and further provides the same convergence rates for each interaction that are not available for BART. Moreover, comprehensive experiments confirm that ANOVA-BART surpasses BART in both accuracy and uncertainty quantification, while also demonstrating its effectiveness in component selection. These results suggest that ANOVA-BART offers a compelling alternative to BART by balancing predictive accuracy, interpretability, and theoretical consistency.

Problem

• Enhancing interpretability of Bayesian Additive Regression Trees
• Decomposing function variability into covariate interactions
• Balancing predictive accuracy with theoretical guarantees

Innovation

• ANOVA-BART extends BART using functional ANOVA decomposition
• Enhances interpretability while preserving Bayesian regression trees
• Provides minimax optimal posterior concentration rates for interactions