Bayes Factor Hypothesis Testing in Meta-Analyses: Practical Advantages and Methodological Considerations

2025-11-27

AI Summary

Traditional meta-analyses rely on *p*-values, limiting quantification of evidence strength, control of Type I error rates, and sequential updating. This paper systematically investigates Bayesian factor (BF) hypothesis testing in meta-analysis, integrating *e*-value theory to enable flexible error-rate control, explicit modeling of prior sensitivity, and implementation via the open-source R package BFpack. We propose a novel BF–*e*-value joint framework that jointly ensures interpretability of statistical evidence, cumulative synthesis across studies, and robustness assessment. Empirical validation is conducted on two applied problems: statistical learning in individuals with language impairments and postoperative seroma incidence following breast cancer surgery—demonstrating reproducible workflows and practical utility. Results indicate substantial improvements in rigor, transparency, and decision-support capability of evidence synthesis, advancing meta-analysis toward probabilistic, dynamic paradigms.

Technology Category

• Search and Optimization: Metareasoning and Metaheuristics
• Machine Learning: Bayesian Learning
• Reasoning under Uncertainty: Other Foundations of Reasoning under Uncertainty

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• User Modeling, Personalization and Recommendation: Metrics for user behavior and evaluating success
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods

Abstract

Bayesian hypothesis testing via Bayes factors offers a principled alternative to classical p-value methods in meta-analysis, particularly suited to its cumulative and sequential nature. Unlike p-values, Bayes factors allow for quantifying support both for and against the existence of an effect, facilitate ongoing evidence monitoring, and maintain coherent long-run behavior as additional studies are incorporated. Recent theoretical developments further show how Bayes factors can flexibly control Type I error rates through connections to e-value theory. Despite these advantages, their use remains limited in the meta-analytic literature. This paper provides a critical overview of their theoretical properties, methodological considerations, such as prior sensitivity, and practical advantages for evidence synthesis. Two illustrative applications are provided: one on statistical learning in individuals with language impairments, and another on seroma incidence following post-operative exercise in breast cancer patients. New tools supporting these methods are available in the open-source R package BFpack.

Problem

• Bayes factors provide an alternative to p-values in meta-analysis
• They quantify support for and against effects in cumulative evidence
• The paper addresses their limited use despite theoretical advantages

Innovation

• Bayes factors quantify support for and against effects
• Bayes factors control Type I error via e-value theory
• Open-source R package BFpack implements these methods