Difficulty-Aware Agent Orchestration in LLM-Powered Workflows

2025-09-13

AI Summary

Existing LLM-based multi-agent systems rely on static or task-level workflows, failing to simultaneously optimize efficiency for simple queries and performance for complex ones, while neglecting the efficiency–performance trade-off across heterogeneous LLMs. Method: We propose the first difficulty-aware dynamic orchestration framework: (i) a variational autoencoder (VAE) enables fine-grained query difficulty estimation; (ii) workflow depth, operator selection, and heterogeneous LLM allocation are dynamically adjusted based on estimated difficulty; and (iii) a modular operator allocator and cost-aware routing strategy support personalized inference path construction. Contribution/Results: Evaluated on six benchmark tasks, our method significantly outperforms state-of-the-art multi-agent systems—achieving higher accuracy while improving inference efficiency. This validates the effectiveness and generality of difficulty-driven dynamic orchestration in heterogeneous LLM environments.

Technology Category

• Search and Optimization: Sampling/Simulation-based Search
• Planning, Routing, and Scheduling: Planning with Language Models
• Multiagent Systems: Adversarial Agents

Application Category

• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

Large Language Model (LLM)-based agentic systems have shown strong capabilities across various tasks. However, existing multi-agent frameworks often rely on static or task-level workflows, which either over-process simple queries or underperform on complex ones, while also neglecting the efficiency-performance trade-offs across heterogeneous LLMs. To address these limitations, we propose Difficulty-Aware Agentic Orchestration (DAAO), a dynamic framework that adapts workflow depth, operator selection, and LLM assignment based on the difficulty of each input query. DAAO comprises three interdependent modules: a variational autoencoder (VAE) for difficulty estimation, a modular operator allocator, and a cost- and performance-aware LLM router. By leveraging heterogeneous LLMs and dynamically tailoring workflows, DAAO enables fine-grained, query-specific reasoning strategies. DAAO outperforms prior multi-agent systems in both accuracy and inference efficiency across six benchmarks. We will release our code and implementation details upon publication.

Problem

• Dynamic workflow adaptation for query difficulty
• Optimizing efficiency-performance trade-offs in LLM systems
• Fine-grained reasoning strategies for heterogeneous queries

Innovation

• Dynamic workflow adaptation based on query difficulty
• VAE-based difficulty estimation and modular operator allocation
• Cost-performance aware routing across heterogeneous LLMs