Think Big, Search Small: Where Capacity Matters in Hierarchical Search Agents?

2026-07-08

AI Summary

This study addresses the optimal allocation of large language model capacity in hierarchical search agents to enhance multi-hop question answering performance. By decoupling the system into three functional roles—task delegation, retrieval execution, and answer generation—the authors conduct capacity-controlled experiments and identify task delegation as the primary performance bottleneck. Guided by this insight, they propose a new architectural paradigm that concentrates model capacity in the delegation module while significantly compressing the executor size. Leveraging role decomposition, capacity scanning, and trajectory distillation, they train a lightweight 1.7B-parameter executor. Experimental results demonstrate that this approach improves Exact Match scores by 4.5–8.6 points; scaling the delegator yields approximately 11-point gains, whereas the executor contributes only about 2.6 points. The lightweight executor maintains competitive accuracy while reducing sub-agent token consumption by 37%, matching the performance of state-of-the-art models.

Abstract

Large language model based search agents increasingly adopt multi-agent architectures in which a main agent decomposes a complex question into sub-queries and dispatches them to parallel sub-agents. However, existing systems instantiate all roles from a single model of identical scale, leaving open how model capacity should be distributed across roles. We factorize hierarchical search into three roles: a delegation role responsible for task decomposition, an execution role responsible for retrieval and evidence extraction, and an answer generation role held fixed as a confound control. We then conduct controlled capacity sweeps along the delegation and execution axes on five multi-hop QA benchmarks. The experiments yield three findings. First, role factorization consistently outperforms a single-agent baseline, improving exact match from 4.5 to 8.6 points across six model scales. Second, capacity sensitivity is asymmetric: scaling the delegation backbone improves EM by ~11 points, whereas scaling the execution sub-agent moves EM by only ~2.6 points, identifying decomposition as the capability bottleneck. Third, a 1.7B-parameter executor trained via quality-filtered trajectory distillation matches a frontier sub-agent in accuracy while consuming 37% fewer sub-agent tokens, advancing the Pareto frontier. These results suggest a concrete recipe for building hierarchical search agents: concentrate capacity at delegation and downsize execution without sacrificing accuracy. Our code is available at https://github.com/QinnanCai0115/role-factorized-search.

Problem

• hierarchical search agents
• model capacity allocation
• multi-agent architecture
• task decomposition
• multi-hop QA

Innovation

• role factorization
• hierarchical search agents
• capacity allocation
• trajectory distillation
• multi-hop QA