๐ค AI Summary
This work addresses the overconfidence of existing large language model (LLM) multi-agent systems when tackling tasks beyond their expertise, stemming from inaccurate self-assessment of capability boundaries. To mitigate this, the authors propose a metacognitive multi-agent framework that integrates verbalized uncertainty expressions, historical capability profiling, inter-agent evaluation, and cybernetic feedback to enable accurate self-assessment of taskโcapability alignment. The framework further incorporates an adaptive task delegation protocol and a capability boundary learning module, facilitating dynamic reassignment of low-confidence tasks. Evaluated on the MetaCog-Eval benchmark, the proposed approach achieves a task accuracy of 82.4%, surpassing the best routing baseline by 8.7%, while reducing API calls by 5% compared to AutoGen and by 34% relative to ensemble voting methods.
๐ Abstract
Multi-agent large language model (LLM) systems have shown promise for solving complex tasks through agent collaboration. However, existing frameworks assign tasks based on predefined roles without considering whether an agent can accurately assess its own competence boundaries, leading to overconfident execution on tasks beyond its expertise. Inspired by metacognition theory from cognitive science, we propose MetaCogAgent, a multi-agent LLM framework where each agent is equipped with a Metacognitive Self-Assessment Unit that evaluates task-capability alignment before execution. The framework introduces three contributions: (1) a self-assessment mechanism that estimates per-task confidence by combining verbalized uncertainty with historical capability profiles; (2) an adaptive delegation protocol that routes low-confidence tasks to better-suited agents through cross-agent evaluation; and (3) a capability boundary learning module that iteratively refines each agent's competence model via cybernetic feedback. Experiments on our constructed MetaCog-Eval benchmark (700 tasks across 5 cognitive dimensions) demonstrate that MetaCogAgent achieves 82.4% task accuracy -- 8.7% above the best routing baseline -- while using 5% fewer API calls than AutoGen and 34% fewer than ensemble voting. Ablation studies confirm that each metacognitive component contributes to overall system performance.