HiveMind: Contribution-Guided Online Prompt Optimization of LLM Multi-Agent Systems

2025-12-06

AI Summary

Quantifying individual agent contributions in large language model (LLM)-based multi-agent systems remains challenging, and inefficient agents lack mechanisms for online optimization. Method: This paper proposes DAG-Shapley, a Shapley-value-based contribution attribution framework that introduces directed acyclic graph (DAG) structures into credit assignment, integrating structural pruning and hierarchical intermediate-result reuse to enhance computational efficiency. It further develops Contribution-Guided Online Prompt Optimization (CG-OPO), a framework for dynamically refining prompts of underperforming agents. Contribution/Results: On a multi-agent stock trading task, DAG-Shapley reduces LLM calls by over 80% compared to full Shapley computation while maintaining comparable attribution accuracy. CG-OPO significantly improves system-level collaboration efficiency over static baselines and strengthens adaptability in complex decision-making scenarios.

Technology Category

• Multiagent Systems: Adversarial Agents
• Cognitive Modeling & Cognitive Systems: Agent Architectures
• Search and Optimization: Learning to Search

Application Category

• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Search and Retrieval-Augmented AI: Agentic search

Abstract

Recent advances in LLM-based multi-agent systems have demonstrated remarkable capabilities in complex decision-making scenarios such as financial trading and software engineering. However, evaluating each individual agent's effectiveness and online optimization of underperforming agents remain open challenges. To address these issues, we present HiveMind, a self-adaptive framework designed to optimize LLM multi-agent collaboration through contribution analysis. At its core, HiveMind introduces Contribution-Guided Online Prompt Optimization (CG-OPO), which autonomously refines agent prompts based on their quantified contributions. We first propose the Shapley value as a grounded metric to quantify each agent's contribution, thereby identifying underperforming agents in a principled manner for automated prompt refinement. To overcome the computational complexity of the classical Shapley value, we present DAG-Shapley, a novel and efficient attribution algorithm that leverages the inherent Directed Acyclic Graph structure of the agent workflow to axiomatically prune non-viable coalitions. By hierarchically reusing intermediate outputs of agents in the DAG, our method further reduces redundant computations, and achieving substantial cost savings without compromising the theoretical guarantees of Shapley values. Evaluated in a multi-agent stock-trading scenario, HiveMind achieves superior performance compared to static baselines. Notably, DAG-Shapley reduces LLM calls by over 80% while maintaining attribution accuracy comparable to full Shapley values, establishing a new standard for efficient credit assignment and enabling scalable, real-world optimization of multi-agent collaboration.

Problem

• Optimizing underperforming agents in LLM multi-agent systems
• Quantifying individual agent contributions for prompt refinement
• Reducing computational cost of Shapley value attribution efficiently

Innovation

• Contribution-Guided Online Prompt Optimization refines agent prompts autonomously
• DAG-Shapley algorithm efficiently quantifies agent contributions using workflow structure
• Hierarchical reuse of intermediate outputs reduces computational costs significantly