Token Complexity of Certifying Stochastic-Oracle Reliability

2026-06-22

AI Summary

This work addresses the problem of reliably distinguishing, with minimal expected interaction cost, between stochastic oracles that meet a target reliability level and those that do not. To this end, it introduces the notion of certification token complexity and constructs a stochastic oracle Turing machine based on the Sequential Probability Ratio Test (SPRT), which accumulates log-likelihood ratio evidence to certify reliability. The study provides the first rigorous characterization of the leading-order behavior of the required token complexity under small error probabilities, establishes an explicit upper bound, and proves that this bound matches the information-theoretic lower bound. Consequently, it fully characterizes the information-theoretic limits of optimal certification cost.

Abstract

Wang~\cite{Wang2026} introduced the Stochastic-Oracle Turing Machine (SOTM) framework and defined token complexity as the minimum expected cost of interacting with a stochastic oracle needed to attain a specified solution quality for a task. This paper develops an analogous notion for certifying the reliability of a stochastic oracle on a given domain. Certification token complexity is the minimum expected token cost required, with controlled error probability, to distinguish oracles that meet a target reliability level from those that fall below a lower reliability threshold. We construct an SPRT-based certification SOTM that queries the oracle, computes binary correctness scores, and stops when the accumulated log-likelihood evidence crosses a decision threshold. The SOTM halts almost surely, satisfies the desired two-sided error guarantee over the reliability regions to be certified, and yields an explicit upper bound on certification token complexity in terms of the reliability thresholds, the error bound, and the expected per-turn token cost. We then establish a matching information-theoretic lower bound: even with adaptive queries, every error-bounded certification SOTM must incur the same leading-order expected token cost as the SPRT-based construction as the prescribed error bound tends to zero. Together, these bounds characterize the leading-order certification token complexity in the small-error regime.

Problem

• certification token complexity
• stochastic oracle
• reliability certification
• expected token cost
• error-bounded verification

Innovation

• certification token complexity
• Stochastic-Oracle Turing Machine
• sequential probability ratio test
• reliability certification
• information-theoretic lower bound