Functional BART with Shape Priors: A Bayesian Tree Approach to Constrained Functional Regression

2025-02-24

AI Summary

This paper addresses nonparametric regression with functional responses and scalar predictors. We propose Functional BART (FBART), the first extension of the Bayesian Additive Regression Trees (BART) framework to functional-response settings, integrating B-spline basis expansions with Bayesian tree partitioning. We innovatively introduce embeddable shape-constrained priors—such as monotonicity and convexity—that rigorously enforce domain knowledge in posterior samples. Furthermore, we establish a novel posterior contraction rate theory adaptive to the unknown smoothness of response curves. FBART employs customized Bayesian backfitting and shape-constrained MCMC sampling. Extensive simulations and real-data applications demonstrate that FBART significantly outperforms existing methods in estimation accuracy and predictive performance, while offering strong interpretability, modeling flexibility, and rigorous theoretical guarantees.

Technology Category

• Machine Learning: Bayesian Learning
• Reasoning under Uncertainty: Relational Probabilistic Models
• Knowledge Representation and Reasoning: Nonmonotonic Reasoning

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Graph embeddings and representation learning for Web-related graphs
• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization

Abstract

Motivated by the remarkable success of Bayesian additive regression trees (BART) in regression modelling, we propose a novel nonparametric Bayesian method, termed Functional BART (FBART), tailored specifically for function-on-scalar regression. FBART leverages spline-based representations for functional responses coupled with a flexible tree-based partitioning structure, effectively capturing complex and heterogeneous relationships between response curves and scalar predictors. To facilitate efficient posterior inference, we develop a customized Bayesian backfitting algorithm. Additionally, we extend FBART by introducing shape constraints (e.g., monotonicity or convexity) on the response curves, enabling enhanced estimation and prediction when prior shape information is available. The use of shape priors ensures that posterior samples respect the specified functional constraints. Under mild regularity conditions, we establish posterior convergence rates for both FBART and its shape-constrained variant, demonstrating rate adaptivity to unknown smoothness. Extensive simulation studies and analyses of two real datasets illustrate the superior estimation accuracy and predictive performance of our proposed methods compared to existing state-of-the-art alternatives.

Problem

• Develops Functional BART for function-on-scalar regression with splines
• Incorporates shape constraints like monotonicity for enhanced estimation
• Proves posterior convergence rates for adaptive smoothness handling

Innovation

• Functional BART with spline-based representations
• Customized Bayesian backfitting algorithm
• Shape constraints for enhanced estimation