R2-Router: A New Paradigm for LLM Routing with Reasoning

2026-02-02

AI Summary

This work addresses a critical limitation in existing large language model (LLM) routing methods, which assume fixed model quality and cost while ignoring the impact of variable output lengths on both. This oversight often leads to the erroneous exclusion of high-capability models under budget constraints. To overcome this, we propose R2-Router, the first routing framework that explicitly incorporates output length budgets into the routing decision, jointly optimizing LLM selection and output length. By employing length-constrained instructions to enforce budget adherence, R2-Router transforms the router from a passive selector into an active, reasoning-aware planner. We further introduce R2-Bench, the first benchmark dataset supporting multi-length-budget routing evaluation. Experiments demonstrate that R2-Router achieves state-of-the-art performance, reducing costs by 4–5× over existing methods while maintaining high output quality.

Technology Category

• Planning, Routing, and Scheduling: Planning with Language Models
• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: (Large) Language Models

Application Category

• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Search and Retrieval-Augmented AI: Large language models for search

Abstract

As LLMs proliferate with diverse capabilities and costs, LLM routing has emerged by learning to predict each LLM's quality and cost for a given query, then selecting the one with high quality and low cost. However, existing routers implicitly assume a single fixed quality and cost per LLM for each query, ignoring that the same LLM's quality varies with its output length. This causes routers to exclude powerful LLMs when their estimated cost exceeds the budget, missing the opportunity that these LLMs could still deliver high quality at reduced cost with shorter outputs. To address this, we introduce R2-Router, which treats output length budget as a controllable variable and jointly selects the best LLM and length budget, enforcing the budget via length-constrained instructions. This enables R2-Router to discover that a powerful LLM with constrained output can outperform a weaker LLM at comparable cost-efficient configurations invisible to prior methods. Together with the router framework, we construct R2-Bench, the first routing dataset capturing LLM behavior across diverse output length budgets. Experiments show that R2-Router achieves state-of-the-art performance at 4-5x lower cost compared with existing routers. This work opens a new direction: routing as reasoning, where routers evolve from reactive selectors to deliberate reasoners that explore which LLM to use and at what cost budget.

Problem

• LLM routing
• output length
• cost-quality tradeoff
• budget constraint
• model selection

Innovation

• LLM routing
• output length budget
• cost-quality tradeoff
• reasoning-based selection
• R2-Router