The Catastrophic Consequences of Agnosticism for Life Searches and a Possible Workaround

2026-05-03

AI Summary

In the search for extraterrestrial life, the use of uninformative priors—due to a lack of prior knowledge about unknown confounding factors—often renders positive signals insufficient as strong evidence. This work proposes an innovative AB testing strategy that partitions observational samples into two groups, leveraging potential differences in underlying biosignature abundance between them to enhance detection power, all while remaining agnostic to prior assumptions about confounders. By avoiding the subjective imposition of upper bounds on interference common in conventional approaches, the method achieves improved inferential rigor. Bayesian factor analysis demonstrates that with a total sample size of \(N_{\text{tot}} = 24\), 24% of outcomes meet the threshold for “strong evidence”; this proportion exceeds 50% when \(N_{\text{tot}} \geq 76\), substantially outperforming existing agnostic methods.

Abstract

Planned and ongoing searches for life, both biological and technological, confront an epistemic barrier concerning false positives - namely, that we don't know what we don't know. The most defensible and agnostic approach is to adopt diffuse (uninformative) priors, not only for the prevalence of life, but also for the prevalence of confounders. We evaluate the resulting Bayes factors between the null and life hypotheses for an idealized experiment with $N_{pos}$ positive labels (biosignature detections) among $N_{tot}$ targets with various priors. Using diffuse priors, the consequences are catastrophic for life detection, requiring at least ${\sim}10^4$ (for some priors ${\sim}10^{13}$) surveyed targets to ever obtain "strong evidence" for life. Accordingly, an HWO-scale survey with $N_{tot}{\sim}25$ would have no prospect of achieving this goal. A previously suggested workaround is to forgo the agnostic confounder prior, by asserting some upper limit on it for example, but we find that the results can be highly sensitive to this choice - as well as difficult to justify. Instead, we suggest a novel solution that retains agnosticism: by dividing the sample into two groups for which the prevalence of life differs, but the confounder rate is global. We show that a $N_{tot}=24$ survey could expect 24% of possible outcomes to produce strong life detections with this strategy, rising to $\geq50$% for $N_{tot}\geq76$. However, AB-testing introduces its own unique challenges to survey design, requiring two groups with differing life prevalence rates (ideally greatly so) but a global confounder rate.

Problem

• agnosticism
• life detection
• false positives
• confounders
• Bayes factors

Innovation

• agnostic priors
• Bayes factor
• confounder rate
• life detection
• AB-testing