GRASP: Gradient Realignment via Active Shared Perception for Multi-Agent Collaborative Optimization

2026-04-01

AI Summary

This work addresses the non-stationarity induced by concurrent policy updates in multi-agent cooperative optimization, which often leads to equilibrium oscillations and impedes convergence. To mitigate this issue, the authors propose the GRASP framework, which introduces an innovative active shared perception mechanism enabling agents to explicitly perceive changes in others’ policies. Leveraging this awareness, agents construct consensus gradients to jointly optimize toward a generalized Bellman equilibrium. Theoretically, the existence and attainability of this equilibrium are established via Kakutani’s fixed-point theorem. Empirically, GRASP demonstrates significantly accelerated convergence and enhanced coordination efficiency on benchmark environments including SMAC and Google Research Football, while exhibiting strong scalability across varying numbers of agents.

Abstract

Non-stationarity arises from concurrent policy updates and leads to persistent environmental fluctuations. Existing approaches like Centralized Training with Decentralized Execution (CTDE) and sequential update schemes mitigate this issue. However, since the perception of the policies of other agents remains dependent on sampling environmental interaction data, the agent essentially operates in a passive perception state. This inevitably triggers equilibrium oscillations and significantly slows the convergence speed of the system. To address this issue, we propose Gradient Realignment via Active Shared Perception (GRASP), a novel framework that defines generalized Bellman equilibrium as a stable objective for policy evolution. The core mechanism of GRASP involves utilizing the independent gradients of agents to derive a defined consensus gradient, enabling agents to actively perceive policy updates and optimize team collaboration. Theoretically, we leverage the Kakutani Fixed-Point Theorem to prove that the consensus direction $u^*$ guarantees the existence and attainability of this equilibrium. Extensive experiments on StarCraft II Multi-Agent Challenge (SMAC) and Google Research Football (GRF) demonstrate the scalability and promising performance of the framework.

Problem

• non-stationarity
• multi-agent collaboration
• equilibrium oscillation
• convergence speed
• passive perception

Innovation

• Active Shared Perception
• Consensus Gradient
• Generalized Bellman Equilibrium
• Multi-Agent Non-stationarity
• Gradient Realignment