Uncertainty Quantification for Multi-level Models Using the Survey-Weighted Pseudo-Posterior

2025-10-10

AI Summary

To address inadequate calibration of both local (e.g., group-level random effects) and global parameters in multilevel Bayesian models under complex survey designs, this paper proposes an improved survey-weighted pseudo-posterior framework with an automated post-processing pipeline—enabling, for the first time, consistent uncertainty calibration for both parameter types within hierarchical structures. The method integrates mixed-effects modeling, weighted likelihood construction, and posterior reweighting calibration, ensuring asymptotic consistency theoretically and seamless compatibility with mainstream Bayesian software computationally. Simulation studies and empirical analysis using the National Survey on Drug Use and Health (NSDUH) demonstrate that the proposed approach substantially improves estimation accuracy and reliability of uncertainty quantification, particularly reducing bias in random-effects inference. The corresponding algorithms are publicly available and integrated into the R package `csSampling`, offering a scalable, reproducible solution for hierarchical Bayesian analysis under complex sampling.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Reasoning under Uncertainty: Relational Probabilistic Models
• Search and Optimization: Sampling/Simulation-based Search

Application Category

• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Semantics and Knowledge: Methods, algorithms and applications for the development of semantic models, knowledge graphs and other forms of structured data models with machine-interpretable semantics

Abstract

Parameter estimation and inference from complex survey samples typically focuses on global model parameters whose estimators have asymptotic properties, such as from fixed effects regression models. We present a motivating example of Bayesian inference for a multi-level or mixed effects model in which both the local parameters (e.g. group level random effects) and the global parameters may need to be adjusted for the complex sampling design. We evaluate the limitations of the survey-weighted pseudo-posterior and an existing automated post-processing method to incorporate the complex survey sample design for a wide variety of Bayesian models. We propose modifications to the automated process and demonstrate their improvements for multi-level models via a simulation study and a motivating example from the National Survey on Drug Use and Health. Reproduction examples are available from the authors and the updated R package is available via github:https://github.com/RyanHornby/csSampling

Problem

• Quantify uncertainty in multi-level models using survey-weighted pseudo-posterior
• Adjust local and global parameters for complex survey designs
• Improve automated methods for Bayesian inference with survey data

Innovation

• Modified automated process for survey-weighted pseudo-posterior
• Enhanced Bayesian multi-level model parameter adjustments
• Improved complex survey design incorporation via simulation