Population-Aware Imitation Learning in Mean-field Games with Common Noise

2026-05-05

AI Summary

This work addresses mean-field games with common noise by proposing a population-aware imitation learning approach designed to either recover the Nash equilibrium or achieve optimal performance within an expert population. By constructing an agent loss that combines behavioral cloning with adversarial divergence, the study establishes, for the first time, finite-sample error bounds for imitation learning in this setting, demonstrating that the proposed loss effectively controls both policy exploitability and performance gap. The analysis further reveals that neglecting population information leads to policy failure. Integrating generalized fictitious play with deep learning, the authors develop a numerical framework for computing population-aware policies. Experiments show that standard non-population-aware strategies fail to capture equilibrium dynamics, whereas the proposed method successfully handles the stochasticity induced by common noise.

Abstract

Mean Field Games (MFGs) provide a powerful framework for modeling the collective behavior of large populations of interacting agents. In this paper, we address the problem of Imitation Learning (IL) in MFGs subject to common noise, where the population distribution evolves stochastically. This stochasticity compels agents to adopt population-aware policies to respond to aggregate shocks. We formulate two distinct learning objectives: recovering a Nash equilibrium and maximizing performance against an expert population. We investigate two imitation proxies: Behavioral Cloning (BC) and Adversarial (ADV) divergence. We then establish finite-sample error bounds showing that minimizing these proxies effectively controls both the policy's exploitability and its performance gap relative to the expert. Furthermore, we propose a numerical framework using generalized Fictitious Play and Deep Learning to compute expert population-aware policies. Through experiments on three environments we demonstrate that standard population-unaware policies fail to capture the equilibrium dynamics. Our results highlight that learning population-aware policies is crucial to avoid being misled by the randomness inherent in common noise.

Problem

• Mean Field Games
• Imitation Learning
• Common Noise
• Population-aware Policies
• Stochastic Dynamics

Innovation

• Population-Aware Imitation Learning
• Mean-Field Games
• Common Noise
• Finite-Sample Error Bounds
• Generalized Fictitious Play