CascadeDebate: Multi-Agent Deliberation for Cost-Aware LLM Cascades

2026-04-14

AI Summary

This work addresses the inefficiency of conventional cascaded large language model (LLM) systems, which often prematurely invoke expensive high-capacity models due to insufficient confidence on ambiguous queries, leading to unnecessary computational costs. To mitigate this, the authors propose a novel cascaded architecture that integrates a lightweight multi-agent negotiation mechanism at each escalation boundary. This mechanism activates consensus-based reasoning only for uncertain queries and is coupled with an online threshold optimizer that dynamically adjusts computational resource allocation. The resulting elastic inference framework alternates between single-model and multi-agent reasoning modes. By embedding multi-agent negotiation directly into LLM cascade layers—a first in the field—the method achieves a 26.75% average improvement over strong baselines across five scientific, medical, and general-knowledge benchmarks, with the online optimizer yielding relative accuracy gains of 20.98%–52.33%, effectively balancing performance and cost efficiency.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Multiagent Systems: Agreement, Argumentation & Negotiation
• Search and Optimization: Learning to Search

Application Category

• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

Cascaded LLM systems coordinate models of varying sizes with human experts to balance accuracy, cost, and abstention under uncertainty. However, single-model tiers at each stage often struggle with ambiguous queries, triggering premature escalations to costlier models or experts due to under-confidence and inefficient compute scaling. CascadeDebate addresses this gap by inserting multi-agent deliberation directly at each tier's escalation boundary. Confidence-based routers activate lightweight agent ensembles only for uncertain cases, enabling consensus-driven resolution of ambiguities internally without invoking higher-cost upgrades. Our unified architecture alternates single-model inference with selective multi-agent deliberation across model scales, culminating in human experts as the final fallback. This design scales test-time compute dynamically according to query difficulty. Across five benchmarks spanning science, medicine, and general knowledge, CascadeDebate outperforms strong single-model cascades and standalone multi-agent systems by up to 26.75 percent. An online threshold optimizer proves essential, boosting accuracy by 20.98 to 52.33 percent relative improvement over fixed policies and enabling elastic adaptation to real-world distributions.

Problem

• LLM cascades
• uncertainty handling
• cost-aware inference
• query ambiguity
• model escalation

Innovation

• multi-agent deliberation
• cost-aware LLM cascades
• confidence-based routing
• dynamic compute scaling
• online threshold optimization