DRF: LLM-AGENT Dynamic Reputation Filtering Framework

2025-09-06

AI Summary

To address the challenges of quantifying agent performance and lacking effective credibility assessment mechanisms in multi-agent systems, this paper proposes the Dynamic Reputation Filtering (DRF) framework. DRF constructs an interactive scoring network that jointly models agent honesty and capability, introduces a dynamic reputation scoring mechanism, and integrates an Upper Confidence Bound (UCB)-driven agent selection strategy. Unlike static or single-metric evaluation approaches, DRF enables online reputation updates and task-adaptive agent filtering. Experiments on logical reasoning and code generation tasks demonstrate that DRF improves task completion quality by 23.6% and collaboration efficiency by 18.4% over baseline methods. Moreover, DRF exhibits strong scalability, making it suitable for large-scale multi-agent coordination scenarios.

Technology Category

• Multiagent Systems: Mechanism Design
• Cognitive Modeling & Cognitive Systems: Agent Architectures
• Intelligent Robots: Multi-Robot Systems

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Economics, Online Markets and Human Computation: Trust and reliance of crowd workers and data experts on GenAI

Abstract

With the evolution of generative AI, multi - agent systems leveraging large - language models(LLMs) have emerged as a powerful tool for complex tasks. However, these systems face challenges in quantifying agent performance and lack mechanisms to assess agent credibility. To address these issues, we introduce DRF, a dynamic reputation filtering framework. DRF constructs an interactive rating network to quantify agent performance, designs a reputation scoring mechanism to measure agent honesty and capability, and integrates an Upper Confidence Bound - based strategy to enhance agent selection efficiency. Experiments show that DRF significantly improves task completion quality and collaboration efficiency in logical reasoning and code - generation tasks, offering a new approach for multi - agent systems to handle large - scale tasks.

Problem

• Quantifying agent performance in multi-agent systems
• Assessing agent credibility and reputation mechanisms
• Enhancing agent selection efficiency for complex tasks

Innovation

• Dynamic reputation filtering for agent performance
• Reputation scoring mechanism for agent honesty
• Upper Confidence Bound strategy for selection efficiency