Formal Verification of Learned Multi-Agent Communication Policies via Decision Tree Distillation

2026-06-17

AI Summary

This work addresses the lack of formal safety guarantees in neural network policies for multi-agent reinforcement learning, which hinders their deployment in safety-critical applications such as drone swarms. The paper proposes the first end-to-end framework that distills learned communication policies into interpretable decision trees and formally verifies them using probabilistic model checking, ensuring that verified safety properties transfer back to the original policy. Key innovations include a compositional verification approach based on pairwise decomposition and union-bound aggregation, and the introduction of discrete VQ-VIB messages to enhance distillation fidelity and verification efficiency. Evaluated on cooperative tasks with 5–7 drones, the framework satisfies all 18 PCTL temporal logic specifications (collision probability 0.3% < 1%), achieves a distillation fidelity of 97.9% ± 1.2%, and incurs a verification bias of at most 0.6 percentage points.

Abstract

Multi-agent reinforcement learning (MARL) enables agents to develop coordination strategies through emergent communication, but neural policies lack the formal safety guarantees required for safety-critical robotic deployment in drone swarms and autonomous vehicle fleets. We present the first end-to-end framework for safety verification of learned multi-agent communication policies through policy abstraction: neural policies are distilled into interpretable decision trees, then formally verified, with empirical validation confirming that verified safety properties transfer to original networks. Our four-stage pipeline consists of domain-specific feature extraction from agent observations, decision tree distillation achieving 97.9% +/- 1.2% fidelity to neural policies, automated translation to PRISM probabilistic model checker specifications with complete feature-to-state-variable correspondence, and compositional verification of Probabilistic Computation Tree Logic (PCTL) properties via pairwise decomposition with union-bound aggregation and empirical neighbor modeling. Evaluating Vector-Quantized Variational Information Bottleneck (VQ-VIB) policies for multi-drone coordination with 5-7 agents, we verify 18 temporal logic properties across safety, liveness, and cooperation, achieving 88.9% property satisfaction with all five safety thresholds satisfied (0.3% collision probability vs. 1% threshold). Monte Carlo validation of original neural policies confirms that verified safety properties transfer with <=0.6 percentage-point deviation (95% CI). Discrete VQ-VIB messages provide +11.6 to +13.6 percentage-point fidelity advantages over continuous methods, enabling 3-4x faster verification. Our framework provides empirically validated safety verification for distilled policy abstractions, serving as a practical bridge between deep MARL and formal safety workflows for multi-robot deployment.

Problem

• Formal Verification
• Multi-Agent Reinforcement Learning
• Safety Guarantees
• Neural Policies
• Multi-Robot Systems

Innovation

• decision tree distillation
• formal verification
• multi-agent reinforcement learning
• probabilistic model checking
• VQ-VIB communication