Every pooling rule has its world: matching probability combination rules to situations and stakes

2026-08-11

AI Summary

This study addresses how to select an appropriate fusion rule for combining two probabilistic assessments of the same binary question, based on their semantic origins and dependency structure. It systematically analyzes the underlying assumptions of various probability fusion methods—such as averaging and odds multiplication—derives the corresponding combination formulas, and validates their performance under matched and mismatched conditions via Monte Carlo simulations. The work clarifies the prerequisite conditions for each fusion rule’s validity, demonstrates that relying solely on binary accuracy obscures differences in probability calibration, proposes a method to preserve shared premises for accurately computing success probabilities across multiple reasoning paths, and proves that pairwise fusion loses information when more than two paths are involved. Experiments confirm that correct application of fusion rules recovers true probabilities accurately, whereas misuse incurs substantial performance degradation.

Abstract

Systems often need to combine two numerical assessments of the same yes/no question. The appropriate formula depends on what the numbers represent and on how the sources are related. Averaging is correct when one of several alternative interpretations applies; multiplying odds is correct when probability reports are based on conditionally independent evidence and a common prior; and probabilities of alternative successful derivations require their dependence or shared evidence to be taken into account. We state the assumptions behind several common combination rules and derive the corresponding combined probabilities. Two groups of Monte Carlo experiments address different questions. First, controlled generating mechanisms verify that the derived rule recovers the correct probability in the situations for which its assumptions hold. Second, the same mechanisms measure the consequences of using a mismatched rule, using logarithmic score and threshold decisions with different costs. Distinct pooling rules can produce the same binary decision at threshold 1/2 while assigning substantially different probabilities, so binary accuracy alone can conceal important differences. We also give probabilistic interpretations of conflicting-evidence rules and show that, for overlapping derivations, retaining the identities of shared uncertain premises permits direct calculation of the probability that at least one derivation is available. Pairwise combination of proof probabilities loses information when there are three or more derivations.

Problem

• probability combination
• pooling rules
• conditional independence
• evidence dependence
• uncertain reasoning

Innovation

• probability pooling
• combination rules
• conditional independence
• shared evidence
• Monte Carlo evaluation