CONCAT: Consensus- and Confidence-Driven Ad Hoc Teaming for Efficient LLM-Based Multi-Agent Systems

2026-05-28

AI Summary

This work addresses the high computational overhead in existing large language model (LLM) multi-agent systems caused by frequent communication, as well as the limited generalizability and high cost of training-dependent optimization approaches. The authors propose a novel training-free collaborative framework that dynamically forms transient agent groups and prunes redundant communication by integrating consensus clustering, confidence-based evaluation, and theory-of-mind-inspired prediction of collaboration utility. Evaluated across three mainstream LLMs and standard benchmark tasks, the method achieves up to a 2.02× improvement in efficiency, with an average latency reduction of 50.1% on Qwen2.5-14B-Instruct, outperforming current training-dependent strategies while maintaining strong task performance.

Abstract

Although large language model (LLM) based multi-agent systems (MAS) show their capability to solve complex tasks and achieve higher performance over single agent systems, they lead to huge computational overheads because of heavy communication between agents. Previous research has made efforts to train a sparse multi-agent graph or fine-tune a planner to orchestrate the workflow better. However, such extra training processes introduce computational costs and limit MAS to specific domains, therefore compromising their generalizability. In this paper, we propose CONCAT, a training-free multi-agent collaboration framework based on CONsensus and Confidence-driven Ad hoc Teaming to efficiently organize agent interactions. Specifically, agents are clustered based on their initial answers, and leaders of each cluster are selected based on the agents' confidence. Then, a heuristic function based on the Theory of Mind is designed to predict the collaboration benefits between every two leaders according to their answers and confidence. Finally, an ad hoc multi-agent network is organized after evicting a percentage of communications based on the predicted benefits. Experiments across three LLMs and three benchmarks show that CONCAT achieves up to 2.02x higher efficiency (accuracy/latency ratio) than LLM-Debate and outperforms training-aware methods such as AgentDropout, while reducing average latency by 50.1% on Qwen2.5-14B-Instruct, without any task-specific training.

Problem

• multi-agent systems
• computational overhead
• communication efficiency
• generalizability
• large language models

Innovation

• training-free
• confidence-driven
• ad hoc teaming
• consensus clustering
• Theory of Mind