Budgeted Act-or-Defer Multi-Agent LLM Deliberation with Local Reliability Bounds

2026-06-28

AI Summary

This work addresses the challenge of dynamically deciding whether a multi-agent large language model system should execute an answer or defer to human review under a limited error budget. The problem is formalized as a budget-constrained “act-or-defer” decision framework, where debate prefixes are mapped to low-dimensional states, and a k-nearest-neighbor lower confidence bound on state-conditional correctness is computed using calibration data. An action is executed only when this bound exceeds a user-specified threshold. The authors introduce a novel conditionally safe assurance framework that decomposes the total error budget into three interpretable components: calibration failure, residual risk, and representation gap, enabling falsifiable diagnostics and task-difficulty-normalized budget allocation. Evaluated across six benchmarks, the method achieves 84% automation rate and 96% execution accuracy using only 9–12% of the error budget, substantially outperforming nine baselines.

Abstract

Multi-agent deliberation among LLMs can improve reasoning, but deployment requires deciding when the current answer is reliable enough to act on and when it should be escalated to human review. We formulate this as budgeted act-or-defer decision making. At each round, the system maps the debate prefix to a low-dimensional state, computes a $k$-nearest-neighbor lower confidence bound on state-conditional correctness using calibration data, and acts only when the bound exceeds a user-specified reliability threshold. The certificate controls wrong actions through the decomposition $β= δ+ α+ \varepsilon_{\mathrm{act}}$, separating calibration failure, residual action risk, and representation gap. The guarantee is conditional, not distribution-free: it relies on a valid local bias envelope and an action-region representation-gap bound, and each assumption is paired with falsification-style diagnostics. Because the same absolute wrong-action budget has different meanings across tasks of different difficulty, we set budgets relative to each task's final-round error using training data only, and evaluate safety by normalized budget usage $\mathrm{WA}/β$. On six benchmarks against nine baselines, the method uses 9--12% of the pre-declared budget on activated datasets, reaching up to 84% automation and 96% acted-on accuracy; on stress-test datasets, it defers rather than forcing unreliable automation. Rather than relying on per-task post-hoc threshold search, the method prospectively converts a user-declared wrong-action budget into an auditable act-or-defer operating point before deployment, under explicitly stated assumptions.

Problem

• multi-agent deliberation
• act-or-defer
• reliability bounds
• budgeted decision making
• LLM reasoning

Innovation

• budgeted decision-making
• multi-agent LLM deliberation
• local reliability bounds
• act-or-defer
• calibrated confidence bounds