RouteProfile: Elucidating the Design Space of LLM Profiles for Routing

2026-04-30

AI Summary

This work addresses the critical yet overlooked role of model capability profiling in large language model (LLM) routing. The authors propose RouteProfile, a framework that formulates LLM profiling as structured information integration over heterogeneous interaction histories. For the first time, it systematically decouples profile design from routing mechanisms and explores the profile design space across four dimensions: organizational form, representation type, aggregation depth, and learning configuration. Experiments demonstrate that structured profiles outperform flat ones, query-level signals surpass domain-level signals, and trainable structured profiles significantly enhance generalization to unseen models. Evaluations across three representative routers consistently validate the superiority of the proposed approach, establishing a foundation for fair comparison and principled design of LLM routing systems.

Abstract

As the large language model (LLM) ecosystem expands, individual models exhibit varying capabilities across queries, benchmarks, and domains, motivating the development of LLM routing. While prior work has largely focused on router mechanism design, LLM profiles, which capture model capabilities, remain underexplored. In this work, we ask: How does LLM profile design affect routing performance across different routers? Addressing this question helps clarify the role of profiles in routing, disentangle profile design from router design, and enable fairer comparison and more principled development of routing systems. To this end, we view LLM profiling as a structured information integration problem over heterogeneous interaction histories. We develop a general design space of LLM profiles, named RouteProfile, along four key dimensions: organizational form, representation type, aggregation depth, and learning configuration. Through systematic evaluation across three representative routers under both standard and new-LLM generalization settings, we show that: (1) structured profiles consistently outperform flat ones; (2) query-level signals are more reliable than coarse domain-level signals; and (3) generalization to newly introduced models benefits most from structured profiles under trainable configurations. Overall, our work highlights LLM profile design as an important direction for future routing research.

Problem

• LLM routing
• model profiling
• capability representation
• design space
• generalization

Innovation

• LLM profiling
• RouteProfile
• structured representation
• router generalization
• information integration