Contract-Based Compositional Shielding for Safe Multi-Agent Reinforcement Learning

2026-06-12

AI Summary

This work addresses the challenge of ensuring global safety in multi-agent reinforcement learning, where individual agents cannot independently guarantee safety due to action validity depending on the dynamic behaviors of others. The authors propose a contract-based compositional shielding mechanism that provides deterministic safety guarantees under decentralized execution: agents collaboratively select local obligation tuples under a shared global LTL safety specification, balancing team optimality and runtime safety. The approach generates local action masks through verifiable local obligation contracts and dynamically optimizes obligation combinations during end-to-end training using a non-stationary multi-armed bandit algorithm, thereby recovering safety-critical collaborative behaviors that would otherwise require explicit coordination. Evaluated across six environments and fifteen algorithmic variants, this method demonstrates, for the first time, team-optimal safe policies without centralized control.

Abstract

Safe coordination problems surface in multi-agent reinforcement learning when global safety cannot be enforced by any agent unilaterally: the admissibility of one agent's action may depend on the dynamics of other agents. Decentralised shields can enforce safety at runtime, but purely factorised permissions often exclude optimal team behaviour that is safe only through coordination. We study deterministic safety guarantees for agents trained and deployed under decentralised execution, recovering team-optimal safe behaviour without centralised runtime control. Agents have a shared global specification $φ$ in the safety fragment of Linear Temporal Logic ($\mathsf{LTL}_{\mathsf{safe}}$ ), and select among tuples of local $\mathsf{LTL}_{\mathsf{safe}}$ obligations whose conjunction implies the global specification $φ$. Each agent may rely on the other agents' local obligations as assumptions because the whole contract tuple is certified simultaneously and allows projection into local action masks. At learning time, a non-stationary multi-armed bandit chooses among a library of local $\mathsf{LTL}_{\mathsf{safe}}$ obligations to select the tuple that optimises team reward, all without forgoing end-to-end safety. We evaluate the approach across 6 environments and 15 algorithmic variants.

Problem

• multi-agent reinforcement learning
• safety
• coordination
• decentralized execution
• Linear Temporal Logic

Innovation

• compositional shielding
• multi-agent reinforcement learning
• LTL_safe
• decentralized safety
• contract-based coordination