LLM-Powered Decentralized Generative Agents with Adaptive Hierarchical Knowledge Graph for Cooperative Planning

2025-02-08

AI Summary

Existing centralized training with decentralized execution (CTDE) frameworks struggle with long-horizon collaborative planning for multi-agent systems in dynamic open-world environments, due to reliance on centralized long-term planning, static collaboration policies, and limited multimodal data processing capabilities. Method: We propose a decentralized generative agent architecture featuring a novel Hierarchical Adaptive Knowledge Graph Memory and Structured Communication System (DAMCS), integrating large language models (LLMs) with multimodal hierarchical knowledge representation to enable experience organization, on-demand information sharing, and dynamic co-evolution of collaboration strategies. Contribution/Results: Evaluated in the Crafter environment, our approach enables two- and six-agent teams to complete tasks in 37% and 26% fewer steps, respectively, achieving 63% and 74% higher sample efficiency than single-agent baselines—significantly outperforming state-of-the-art MARL and LLM-based methods.

Technology Category

• Multiagent Systems: Multiagent Learning
• Cognitive Modeling & Cognitive Systems: Agent Architectures
• Machine Learning: Large Multimodal Models (LMMs)

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Developing intelligent agents for long-term cooperation in dynamic open-world scenarios is a major challenge in multi-agent systems. Traditional Multi-agent Reinforcement Learning (MARL) frameworks like centralized training decentralized execution (CTDE) struggle with scalability and flexibility. They require centralized long-term planning, which is difficult without custom reward functions, and face challenges in processing multi-modal data. CTDE approaches also assume fixed cooperation strategies, making them impractical in dynamic environments where agents need to adapt and plan independently. To address decentralized multi-agent cooperation, we propose Decentralized Adaptive Knowledge Graph Memory and Structured Communication System (DAMCS) in a novel Multi-agent Crafter environment. Our generative agents, powered by Large Language Models (LLMs), are more scalable than traditional MARL agents by leveraging external knowledge and language for long-term planning and reasoning. Instead of fully sharing information from all past experiences, DAMCS introduces a multi-modal memory system organized as a hierarchical knowledge graph and a structured communication protocol to optimize agent cooperation. This allows agents to reason from past interactions and share relevant information efficiently. Experiments on novel multi-agent open-world tasks show that DAMCS outperforms both MARL and LLM baselines in task efficiency and collaboration. Compared to single-agent scenarios, the two-agent scenario achieves the same goal with 63% fewer steps, and the six-agent scenario with 74% fewer steps, highlighting the importance of adaptive memory and structured communication in achieving long-term goals. We publicly release our project at: https://happyeureka.github.io/damcs.

Problem

• Enhance multi-agent cooperation in dynamic environments.
• Address scalability and flexibility in MARL frameworks.
• Optimize long-term planning using adaptive knowledge graphs.

Innovation

• LLM-powered decentralized generative agents
• Adaptive hierarchical knowledge graph memory
• Structured communication protocol for cooperation