Learning Latency-Aware Orchestration for Multi-Agent Systems

2026-07-14

AI Summary

This work addresses the challenge in multi-agent systems where existing orchestration methods struggle to balance inference efficiency and task accuracy due to neglecting critical-path latency. To this end, the authors propose LAMaS, a novel framework that integrates critical-path-aware constraint optimization during training to learn low-latency execution graphs, and employs a lightweight adaptive controller at inference time to dynamically prune redundant agent interactions. LAMaS is the first to incorporate critical-path-aware credit assignment into multi-agent orchestration, synergistically combining it with runtime dynamic pruning for joint optimization of latency and accuracy. Experiments demonstrate that LAMaS reduces end-to-end latency by over 50% on average across four benchmarks while maintaining comparable or superior accuracy, and exhibits strong modularity and transferability.

Abstract

Multi-agent systems (MAS) coordinate multiple LLM-powered agents through structured workflows, gaining reasoning power but incurring high inference latency from multi-step execution and repeated model invocations. Existing orchestration methods primarily optimize task performance and inference cost, leaving latency largely unaddressed. In MAS, end-to-end latency is governed by the critical execution path, so reducing total cost alone does not reliably reduce latency. Moreover, optimizing latency while preserving accuracy remains non-trivial: naive latency optimization can misassign operator-level credit and degrade task accuracy. To address this gap, we propose Latency-Aware Multi-agent System (LAMaS), a latency-aware orchestration framework for learning-based multi-agent systems. LAMaS addresses this challenge at two levels: at training time, it learns latency-aware execution graphs through constrained optimization with critical-path-aware credit assignment; at inference time, since a graph committed at training time cannot exploit runtime evidence, it complements graph construction with a lightweight controller that adaptively eliminates redundant future agent interactions as execution unfolds. Experiments on four benchmarks show that LAMaS achieves the best latency among evaluated learning-based MAS baselines, reducing end-to-end latency by over 50\% while maintaining competitive or better accuracy. LAMaS is also modular and transfers to other MAS with minimal changes, consistently yielding latency reductions.

Problem

• multi-agent systems
• inference latency
• latency optimization
• critical path
• task accuracy

Innovation

• latency-aware orchestration
• multi-agent systems
• critical-path optimization
• adaptive execution
• LLM-powered agents