Extending NGU to Multi-Agent RL: A Preliminary Study
This work addresses the challenge of coordinated exploration in multi-agent reinforcement learning (MARL) under sparse rewards. We propose the first extension of the Never Give Up (NGU) algorithm to the multi-agent setting, building upon multi-agent DQN and integrating an episode-level novelty-based intrinsic reward mechanism. Our method systematically investigates three key design choices: (i) a shared replay buffer across agents, (ii) cross-agent novelty sharing—parameterized by a tunable *k*—and (iii) heterogeneous β parameters for individualized intrinsic reward scaling. Experiments in the PettingZoo *simple_tag* environment demonstrate that a shared replay buffer significantly improves both performance and training stability; novelty sharing with *k* = 1 enhances exploratory behavior; and heterogeneous β yields no measurable benefit. The core contribution is the empirical validation of synergistic intrinsic motivation and experience sharing, establishing a scalable and robust NGU-inspired paradigm for tackling sparse-reward problems in MARL.