A Theory of Multi-Agent Generative Flow Networks

2025-09-24

AI Summary

Existing Generative Flow Networks (GFlowNets) lack a theoretical framework for multi-agent collaborative modeling. Method: This paper introduces Multi-Agent GFlowNets (MA-GFlowNets), the first unified framework enabling joint action-sequence generation where sampling probabilities are proportional to a global reward function. We propose a local-global training principle and design four algorithmic architectures that reconcile centralized training with decentralized execution—guaranteeing convergence. Key technical innovations include flow-matching loss, stochastic policy modeling, joint flow network architecture, and conditionally independent flow network design. Results: Experiments demonstrate that MA-GFlowNets significantly outperform reinforcement learning and MCMC baselines in both sample quality and sampling efficiency, validating its theoretical rigor and empirical effectiveness across cooperative multi-agent tasks.

Technology Category

• Multiagent Systems: Multiagent Learning
• Machine Learning: Imitation Learning & Inverse Reinforcement Learning
• Cognitive Modeling & Cognitive Systems: Agent Architectures

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Social Networks and Social Media: Generative AI / large language models and their impact on social systems
• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs

Abstract

Generative flow networks utilize a flow-matching loss to learn a stochastic policy for generating objects from a sequence of actions, such that the probability of generating a pattern can be proportional to the corresponding given reward. However, a theoretical framework for multi-agent generative flow networks (MA-GFlowNets) has not yet been proposed. In this paper, we propose the theory framework of MA-GFlowNets, which can be applied to multiple agents to generate objects collaboratively through a series of joint actions. We further propose four algorithms: a centralized flow network for centralized training of MA-GFlowNets, an independent flow network for decentralized execution, a joint flow network for achieving centralized training with decentralized execution, and its updated conditional version. Joint Flow training is based on a local-global principle allowing to train a collection of (local) GFN as a unique (global) GFN. This principle provides a loss of reasonable complexity and allows to leverage usual results on GFN to provide theoretical guarantees that the independent policies generate samples with probability proportional to the reward function. Experimental results demonstrate the superiority of the proposed framework compared to reinforcement learning and MCMC-based methods.

Problem

• Developing theoretical framework for multi-agent generative flow networks
• Enabling collaborative object generation through joint actions
• Providing theoretical guarantees for reward-proportional sampling

Innovation

• Proposes multi-agent generative flow networks theory framework
• Introduces four algorithms for centralized and decentralized training
• Leverages local-global principle for theoretical reward guarantees