🤖 AI Summary
This study addresses the issues of counting scale drift and decision inconsistency arising from the coupling of interpretation and aggregation in multi-source evidence reasoning. We propose a decoupling framework that constructs an evidence quadruplet interface and introduces calibrated log-likelihood ratio pooling to achieve generic rectification at the arithmetic level, thereby effectively separating evidence interpretation from decision aggregation. By integrating sequential encoders with tree ensemble models, the proposed method achieves an AUPRC of 0.921 on longitudinal corpora, significantly outperforming handcrafted baselines (0.805). These results validate the effectiveness of the separation architecture in enhancing the reliability of multi-source reasoning, offering a novel paradigm for complex evidence aggregation.
📝 Abstract
Systems that ask a language model to reach a conclusion from many sources usually concatenate them into one prompt. This conflates two operations with different requirements. Interpreting a source rewards capacity and context. Combining interpretations rewards fixed arithmetic, comparability across instances, and the option to return nothing. Once separated, the design problem becomes the interface between them. We propose a four-field evidence tuple (hypothesis, reliability bucket, rationale, provenance) and show that fixing it determines both halves. The separation also reveals a failure mode in how such systems combine, which we call count-scale drift. Thresholding a sum of unnormalized weights is exactly posterior thresholding, but at an operating point that slides with the number of sources consulted. The slide grows with reader reliability. When source reliabilities differ, the vote rule and the posterior order instances differently, and no threshold reconciles them. Pooling calibrated log-likelihood ratios addresses both problems. The fix is arithmetic rather than architectural, and applies to a class of rules beyond language models: score-summing triage engines, diagnostic panels scored by counting positives, and additive multi-signal detectors. We then instantiate the principle twice on one longitudinal corpus, once after outcomes resolve and once before. The same partition helps in both, at different granularities: over reading in the first, over learning capacity in the second. There, a small sequence encoder on an easy auxiliary objective plus a tree ensemble carrying the censored survival loss reaches 0.921 AUPRC against 0.805 for a hand-crafted baseline. We separate what transfers from what must be re-estimated per domain, and state five predictions that would falsify the framework, three negative results, and which comparisons remain confounded.