To Vary or Not To Vary: A Flexible Empirical Bayes Factor for Testing Variance Components

2025-08-02

AI Summary

Classical variance-component tests for random effects suffer from boundary issues due to non-negativity constraints on variance parameters, leading to invalid asymptotic distributions and poor inferential performance. Method: We propose a flexible empirical Bayes factor (EBF) approach that tests the joint nullity of all random effects—thereby circumventing boundary problems—and employs an empirical Bayes framework to estimate the random-effects distribution automatically, without subjective prior specification. Leveraging the Savage–Dickey density ratio, EBF conducts hypothesis testing within a single full model, avoiding multi-model fitting and prior sensitivity. Contribution/Results: EBF is universally applicable to generalized linear mixed models, spatial random-effects models, dynamic structural equation models, and nonlinear mixed-effects models. Systematic evaluation on synthetic data demonstrates its high statistical power and robustness, substantially enhancing flexibility, generality, and practicality in detecting random effects within complex modeling frameworks.

Technology Category

• Reasoning under Uncertainty: Relational Probabilistic Models
• Machine Learning: Bayesian Learning
• Search and Optimization: Mixed Discrete/Continuous Search

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods
• User Modeling, Personalization and Recommendation: Metrics for user behavior and evaluating success

Abstract

Random effects are the gold standard for capturing structural heterogeneity in data, such as spatial dependencies, individual differences, or temporal dependencies. However, testing for their presence is challenging, as it involves a variance component constrained to be non-negative -- a boundary problem. This paper proposes a flexible empirical Bayes factor (EBF) for testing random effects. Rather than testing whether a variance component is zero, the EBF tests the equivalent hypothesis that all random effects are zero. Crucially, it avoids manual prior specification based on external knowledge, as the distribution of random effects is part of the model's lower level and estimated from the data -- yielding an "empirical" Bayes factor. The EBF uses a Savage-Dickey density ratio, allowing all random effects to be tested using only the full model fit. This eliminates the need to fit multiple models with different combinations of random effects. Simulations on synthetic data evaluate the criterion's general behavior. To demonstrate its flexibility, the EBF is applied to generalized linear crossed mixed models, spatial random effects models, dynamic structural equation models, random intercept cross-lagged panel models, and nonlinear mixed effects models.

Problem

• Testing variance components for random effects presence
• Avoiding manual prior specification in Bayes factor
• Flexible application across diverse statistical models

Innovation

• Flexible empirical Bayes factor for random effects
• Avoids manual prior specification using data
• Uses Savage-Dickey ratio for full model testing