Reward-Guided Autoregressive Graph Generation for Efficient Multi-Agent Communication Topology Design

2026-08-20

AI Summary

本文通过引入奖励引导的自回归图生成方法解决多智能体系统通信拓扑设计中效率低下的问题，减少了20.5%的token消耗同时保持了任务准确性。

Abstract

LLM-based Multi-Agent Systems (MAS) achieve strong performance on complex reasoning tasks by coordinating multiple agents, but at the cost of substantial token consumption. Recent work on automatic topology design, ARG-Designer, has reframed this problem as autoregressive graph generation. However, its training objective provides no explicit incentive for the model to generate sparse and efficient topologies. We address this limitation by introducing a Reward-Guided Autoregressive Graph Generation (RGA-Designer) inspired by Reinforcement Learning from Human Feedback (RLHF). We train a reward model that jointly captures task correctness and structural compactness, and then fine-tune the pretrained graph generator using the reward model as feedback. Our method preserves task accuracy at the level of ARG-Designer while reducing token consumption by an average of 20.5%.

Problem

• Multi-Agent Systems
• Topology Design
• Token Consumption
• Autoregressive Graph Generation
• Sparse Topologies

Innovation

• Reward-Guided Autoregressive Graph Generation
• Reinforcement Learning from Human Feedback
• Structural Compactness
• Token Consumption