Meta-Router: Bridging Gold-standard and Preference-based Evaluations in Large Language Model Routing

2025-09-29

AI Summary

Addressing the scarcity of high-quality supervision signals in LLM routing—where gold-standard annotations are costly and preference data suffer from inherent biases—this paper proposes a causal inference–driven router training framework. Methodologically, it formalizes preference bias as the Conditional Average Treatment Effect (CATE) and achieves bias correction and joint optimization of two heterogeneous supervision sources—expert scores and LLM-as-a-judge preferences—via causal identification and distribution alignment. The approach integrates treatment mechanism modeling with imbalance-aware correction techniques to enable end-to-end robust training. Experiments across multiple language tasks demonstrate that the proposed router significantly improves decision accuracy while reducing average inference cost without compromising response quality. It consistently outperforms existing baselines across all evaluation metrics.

Technology Category

• Machine Learning: Learning Preferences or Rankings
• Planning, Routing, and Scheduling: Planning with Language Models
• Search and Optimization: Learning to Search

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

In language tasks that require extensive human--model interaction, deploying a single "best" model for every query can be expensive. To reduce inference cost while preserving the quality of the responses, a large language model (LLM) router selects the most appropriate model from a pool of candidates for each query. A central challenge to training a high-quality router is the scarcity of reliable supervision. Gold-standard data (e.g., expert-verified labels or rubric-based scores) provide accurate quality evaluations of LLM responses but are costly and difficult to scale. In contrast, preference-based data, collected via crowdsourcing or LLM-as-a-judge systems, are cheaper and more scalable, yet often biased in reflecting the true quality of responses. We cast the problem of LLM router training with combined gold-standard and preference-based data into a causal inference framework by viewing the response evaluation mechanism as the treatment assignment. This perspective further reveals that the bias in preference-based data corresponds to the well-known causal estimand: the conditional average treatment effect. Based on this new perspective, we develop an integrative causal router training framework that corrects preference-data bias, address imbalances between two data sources, and improve routing robustness and efficiency. Numerical experiments demonstrate that our approach delivers more accurate routing and improves the trade-off between cost and quality.

Problem

• Reducing LLM inference costs while maintaining response quality
• Correcting bias in preference-based data for router training
• Integrating gold-standard and preference data via causal inference

Innovation

• Causal framework combines gold-standard and preference data
• Corrects bias in preference data using causal inference
• Improves routing accuracy and cost-quality tradeoff