Calibrated Bayes analysis of cluster-randomized trials

2025-11-25

AI Summary

In cluster randomized trials (CRTs), conventional Bayesian estimators—such as regression coefficients—lack clear causal interpretation under model misspecification or informative cluster sizes. Method: We propose a target-estimand-aligned calibrated Bayesian approach grounded in a hierarchical Bayesian framework, incorporating covariate adjustment, Bayesian nonparametric working models, and calibration priors to ensure frequentist coverage of posterior inference even under misspecification. The method directly targets interpretable causal parameters: the cluster-level average treatment effect (cluster-ATE) and the individual-level average treatment effect (individual-ATE). Robust estimation is achieved via posterior summarization over draws, accommodating continuous, binary, and count outcomes. Contribution/Results: Simulation studies demonstrate that the proposed method accurately estimates both ATEs across diverse misspecification scenarios while maintaining nominal coverage probabilities. It substantially enhances the causal interpretability and robustness of Bayesian analysis in CRTs.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Reasoning under Uncertainty: Causality
• Cognitive Modeling & Cognitive Systems: Adaptive Behavior

Application Category

• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

In cluster-randomized trials (CRTs), entire clusters of individuals are randomized to treatment, and outcomes within a cluster are typically correlated. While frequentist approaches are standard practice for CRT analysis, Bayesian methods have emerged as a strong alternative. Previous work has investigated the use of Bayesian hierarchical models for continuous, binary, and count outcomes in CRTs, but these approaches focus on model-based treatment effect coefficients as the target estimands, which may have ambiguous interpretation under model misspecification and informative cluster size. In this article, we introduce a calibrated Bayesian procedure for estimand-aligned analysis of CRTs even in the presence of potentially misspecified models. We propose estimators targeting both the cluster-average treatment effect (cluster-ATE) and individual-average treatment effect (individual-ATE), particularly in scenarios with informative cluster sizes. We additionally explore strategies for summarizing the posterior samples that can achieve the frequentist coverage guarantee even under working model misspecification. We provide simulation evidence to demonstrate the model-robustness property of the proposed Bayesian estimators in CRTs, and further investigate the impact of covariate adjustment as well as the use of more flexible Bayesian nonparametric working models.

Problem

• Developing calibrated Bayesian methods for cluster-randomized trials
• Addressing model misspecification in treatment effect estimation
• Targeting cluster-average and individual-average treatment effects

Innovation

• Calibrated Bayesian procedure for estimand-aligned CRT analysis
• Targets cluster-average and individual-average treatment effects
• Achieves frequentist coverage under model misspecification