Task-Aware LLM Routing with Multi-Level Task-Profile-Guided Data Synthesis for Cold-Start Scenarios

2026-04-10

AI Summary

This work addresses the poor generalization of existing large language model (LLM) routing systems in cold-start scenarios, where in-domain training data is unavailable. To overcome this limitation, the authors propose TRouter, a novel framework that first generates synthetic question-answer pairs aligned with the target test distribution through a multi-level task-profile-guided data synthesis approach. Building upon this, TRouter introduces a task-type-aware routing mechanism that incorporates a latent task-type variable and leverages prior regularization to enhance routing accuracy. Experimental results demonstrate that TRouter significantly mitigates the cold-start problem across multiple benchmarks and achieves efficient and accurate model routing under both supervised and zero-shot settings.

Technology Category

• Planning, Routing, and Scheduling: Planning with Language Models
• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: (Large) Language Models

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Search and Retrieval-Augmented AI: Large language models for search
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Large language models (LLMs) exhibit substantial variability in performance and computational cost across tasks and queries, motivating routing systems that select models to meet user-specific cost-performance trade-offs. However, existing routers generalize poorly in cold-start scenarios where in-domain training data is unavailable. We address this limitation with a multi-level task-profile-guided data synthesis framework that constructs a hierarchical task taxonomy and produces diverse question-answer pairs to approximate the test-time query distribution. Building on this, we introduce TRouter, a task-type-aware router approach that models query-conditioned cost and performance via latent task-type variables, with prior regularization derived from the synthesized task taxonomy. This design enhances TRouter's routing utility under both cold-start and in-domain settings. Across multiple benchmarks, we show that our synthesis framework alleviates cold-start issues and that TRouter delivers effective LLM routing.

Problem

• LLM routing
• cold-start
• task generalization
• data scarcity
• cost-performance trade-off

Innovation

• LLM routing
• cold-start
• task taxonomy
• data synthesis
• task-aware