CoDiMAD: Diffusion-Based Privileged Distillation for Communication-Free Multi-Robot Coordination

2026-07-10

AI Summary

This work addresses the challenge of policy degradation in decentralized multi-robot coordination under communication constraints and partial observability, where multimodal action distributions lead to suboptimal behavior. To tackle this, the authors propose a three-stage privileged distillation framework: first, a global oracle policy is trained using MAPPO and used to generate an offline dataset pairing local observations with oracle actions; then, a conditional denoising diffusion model distills this oracle knowledge into decentralized agents that rely solely on local observations. This approach pioneers the use of diffusion models in multi-agent privileged distillation, effectively recovering multimodal action distributions and circumventing the mode-averaging pitfalls of deterministic distillation, with accompanying theoretical justification. Experiments demonstrate significant performance gains over both direct local MARL and deterministic distillation baselines across three cooperative tasks, yielding more decisive and coordinated communication-free control.

Abstract

Decentralized multi-robot coordination under partial observability remains challenging, especially in communication-free settings where agents must act solely from local sensor observations. Privileged policy distillation provides a promising approach by transferring knowledge from a globally informed oracle to sensor-constrained students. However, in multi-agent systems, the same local observation may correspond to multiple global configurations requiring qualitatively different cooperative actions, making the conditional action distribution inherently multi-modal. Standard deterministic distillation collapses these modes to their mean, often yielding invalid or hesitant actions. To address this issue, we propose CoDiMAD, a three-stage framework that trains a privileged oracle with MAPPO, constructs an offline dataset of local-observation-oracle-action pairs, and distills the oracle into decentralized students parameterized as conditional denoising diffusion probabilistic models. By approximating the conditional oracle-action distribution through the diffusion reverse process, CoDiMAD samples decisive actions from coherent coordination modes rather than averaging across them. Theoretical analysis characterizes the mode-averaging failure of deterministic distillation and the distributional recovery property of diffusion-based distillation. Experiments on three cooperative tasks show that CoDiMAD consistently outperforms direct local MARL and deterministic distillation baselines. The source code will be made publicly available upon acceptance.

Problem

• multi-robot coordination
• partial observability
• privileged distillation
• multi-modal action distribution
• communication-free

Innovation

• diffusion-based distillation
• privileged policy distillation
• multi-agent coordination
• communication-free
• conditional denoising diffusion