Agentic Systems as Boosting Weak Reasoning Models

2026-05-13

AI Summary

This work investigates how collaborative mechanisms can enhance the overall reasoning performance of weaker language models to rival that of substantially stronger ones. The authors propose a verifier-augmented committee search framework that, during inference, integrates proposers, critics, and comparators to identify correct solutions from multiple sampled candidates. The approach formalizes reasoning enhancement along four dimensions: proposal coverage, local identifiability, progress, and diversity, and crucially argues that local reliability signals—such as execution traces, formal proofs, type checking, test cases, and constraint solving—cannot be adequately replaced by mere sampling. Evaluated on SWE-bench Verified, a single GPT-5.4 nano model augmented with this method achieves a 76.4% task resolution rate, markedly surpassing its baseline performance of 67.0% and approaching the theoretical upper bound of 79.0%, thereby matching the performance of advanced models such as Gemini 3 Pro and Claude Opus 4.5.

Abstract

Can a committee of weak reasoning-model calls reach the performance of much stronger models? We study verifier-backed committee search as inference-time boosting for reasoning language models. The mechanism is not simply that ``more agents help'': samples expose latent correct solutions, while critics and comparators must recover them without access to the hidden verifier. We formalize this view by separating proposal coverage, local identifiability, progress, and diversity. We prove that coverage can be amplified by repeated sampling, but cannot by itself create useful critics or comparators; reliable amplification requires an additional local soundness signal, such as execution, proof checking, type checking, tests, or constraint solving. We give rank-based bounds showing when local selection errors compose into reliable trajectories, and characterize the proposer-side ceiling: oracle best-of-\(k\) converges only to the mass of task slices on which the proposal system assigns nonzero useful probability. Empirically, on SWE-bench Verified, a single \texttt{GPT-5.4 nano} proposal solves \(67.0\%\) of tasks. Using the same nano model, our critic--comparator orchestration reaches \(76.4\%\) with \(k=8\) proposals, matching the standalone performance of \texttt{Gemini 3 Pro} and \texttt{Claude Opus 4.5} Thinking and approaching the \(79.0\%\) oracle best-of-\(8\) upper bound. Thus, many correct patches are already present in weak-model proposal pools; the main challenge is selecting them. The remaining failures are mostly proposal-coverage failures, indicating shared blind spots that stronger selection alone cannot close.

Problem

• agentic systems
• weak reasoning models
• committee search
• proposal coverage
• inference-time boosting

Innovation

• verifier-backed committee search
• inference-time boosting
• weak reasoning models
• proposal coverage
• local soundness signal