RouteGuard: Certifying Routing Gain in LLM Multi-Agent Systems When Complementarity Is Not Enough

2026-08-05

AI Summary

Existing routing mechanisms struggle to reliably determine—prior to deployment—whether multi-agent large language model systems genuinely benefit from routing, as reliance solely on advisor complementarity or AUC often leads to erroneous conclusions. This work proposes RouteGuard, a novel framework that decouples routing gain from AUC for the first time by decomposing the gain into a policy improvement term π and a conditional error gap Δ_E, and introduces a conditional regret functional Φ to construct certifiable performance intervals under finite-sample settings. The method establishes sharp bounds matching the Le Cam lower bound, reveals a robustness phase transition phenomenon, and validates its efficacy through workload-aware clustering-based resampling and preregistered semi-synthetic experiments. Evaluated on RouterBench and OpenRCA benchmarks, RouteGuard accurately identifies effective routing scenarios, avoids spurious certification of redundant advisors, and demonstrates well-calibrated reliability.

Abstract

Multi-agent LLM systems route among model-backed advisors, yet a deployer rarely knows before shipping whether routing will help at all. Prevailing routers optimize a gate's AUC and presume that advisor complementarity suffices. We show that neither determines the deployable gain. We introduce RouteGuard, a deployment-certification framework. Routing gain decomposes as $G = πΔ_E$, and the achievable gain is governed by a conditional-regret functional $Φ$, not by AUC. A finite-sample certification bracket comes with a matching Le Cam lower bound, constant-sharp over the fixed-activity class, and a robustness phase transition. On two benchmarks the framework acts as a guardrail. On RouterBench (11 cross-family models) the verdict depends on the sampling unit: the protocol certifies a gain over GPT-4 under prompt-level sampling and withholds it under workload-cluster resampling, because the gain rests on 3 of 86 workload cells. On OpenRCA (three Gemini advisors) the advisors are statistically redundant: the realized oracle sits at or below the independence baseline in all pools we tested (221 RouterBench pools and three OpenRCA distributions), so the protocol correctly refuses to certify. A pre-registered semi-synthetic control confirms calibration: the protocol certifies a genuine gain once $m \ge m^\star$ and does not certify a true null. Code and frozen artifacts will be released with the published version.

Problem

• routing gain
• multi-agent LLM systems
• deployment certification
• advisor complementarity
• conditional regret

Innovation

• RouteGuard
• routing gain certification
• conditional regret
• multi-agent LLM systems
• deployment validation