Experience as a Compass: Multi-agent RAG with Evolving Orchestration and Agent Prompts

2026-04-01

AI Summary

Existing multi-agent RAG approaches rely on static behaviors and fixed orchestration, limiting their ability to handle diverse, multi-hop complex tasks. This work proposes HERA, a framework that hierarchically coordinates agents by globally optimizing query-driven agent topologies and locally evolving role-aware, dual-axis (action and behavior) prompts. HERA integrates experience-driven dynamic orchestration, reward-guided sampling, experience accumulation, and credit assignment to enable self-organized collaboration among agents. Evaluated across six knowledge-intensive benchmarks, HERA achieves an average performance gain of 38.69%, demonstrating strong generalization, high token efficiency, and the emergence of sparse yet effective collaborative networks.

Abstract

Multi-agent Retrieval-Augmented Generation (RAG), wherein each agent takes on a specific role, supports hard queries that require multiple steps and sources, or complex reasoning. Existing approaches, however, rely on static agent behaviors and fixed orchestration strategies, leading to brittle performance on diverse, multi-hop tasks. We identify two key limitations: the lack of continuously adaptive orchestration mechanisms and the absence of behavior-level learning for individual agents. To this end, we propose HERA, a hierarchical framework that jointly evolves multi-agent orchestration and role-specific agent prompts. At the global level, HERA optimizes query-specific agent topologies through reward-guided sampling and experience accumulation. At the local level, Role-Aware Prompt Evolution refines agent behaviors via credit assignment and dual-axes adaptation along operational and behavioral principles, enabling targeted, role-conditioned improvements. On six knowledge-intensive benchmarks, HERA achieves an average improvement of 38.69\% over recent baselines while maintaining robust generalization and token efficiency. Topological analyses reveal emergent self-organization, where sparse exploration yields compact, high-utility multi-agent networks, demonstrating both efficient coordination and robust reasoning.

Problem

• Multi-agent RAG
• adaptive orchestration
• agent behavior learning
• multi-hop reasoning
• role-specific prompting

Innovation

• Multi-agent RAG
• Evolving Orchestration
• Prompt Evolution
• Role-Aware Adaptation
• Hierarchical Framework