NeuroMAS: Multi-Agent Systems as Neural Networks with Joint Reinforcement Learning

2026-05-15

AI Summary

Traditional multi-agent language systems rely on handcrafted workflows and communication protocols, making them difficult to train and scale. This work proposes a neural-network-inspired multi-agent architecture in which large language models serve as nodes and textual messages act as signal-carrying edges. Through joint reinforcement learning, agents achieve adaptive role specialization and effective coordination. The architecture is structure-aware yet role-agnostic, enabling progressive system growth and shifting the design paradigm from workflow engineering to neural architecture design. Experiments demonstrate that the proposed approach significantly outperforms existing baselines both in inference and after training. Furthermore, the results reveal path dependence in organizational scaling, showing that incremental training substantially enhances parameter efficiency and performance in large-scale systems.

Abstract

Multi-agent language systems are often built as hand-designed workflows, where agents are assigned semantic roles and communication protocols are specified in advance. We propose NeuroMAS, a method that first treats a multi-agent language system as a trainable and scalable neural-network-like architecture with LLM agents as nodes and intermediate textual signals as edges. In NeuroMAS, agent nodes are role-free but structure-aware: the topology only determines how information can flow in general, while reinforcement learning training determines how nodes communicate, specialize, and coordinate. This formulation shifts multi-agent design from workflow engineering toward architecture design, where depth, width, connectivity, and growth protocol become scalable sources of capability. Further, we provide a theoretical perspective showing why such modular textual computation is more parameter-efficient when tasks admit hierarchical decompositions. Experiments show that NeuroMAS improves significantly over both inference-time and trained multi-agent baselines. We further find that organizational scaling is path-dependent: larger systems can be challenging to train from scratch, but become feasible when grown progressively from smaller trained systems. These results suggest that learned neural multi-agent systems are a promising scaling axis for LLMs.

Problem

• multi-agent systems
• language models
• workflow design
• scalability
• coordination

Innovation

• NeuroMAS
• multi-agent systems
• neural architecture
• reinforcement learning
• modular textual computation