Safe and Generalizable Hierarchical Multi-Agent RL via Constraint Manifold Control

2026-06-22

AI Summary

This work addresses the challenge of achieving both rigorous safety guarantees and efficient coordination in safety-critical multi-agent systems. The authors propose a hierarchical multi-agent reinforcement learning framework in which a low-level controller enforces hard safety constraints through constrained manifold control under mild assumptions, while a high-level policy learns to coordinate agents effectively. This approach represents the first integration of constrained manifold control with hierarchical reinforcement learning in a multi-agent setting, offering provable safety, stable training dynamics, and strong generalization across varying numbers of agents and obstacle configurations. Experimental results demonstrate that the system maintains nearly 100% safety compliance while achieving competitive task performance.

Abstract

Multi-agent systems are widely used in safety-critical applications that require coordinated behavior under strict safety constraints. Existing approaches face a fundamental trade-off: learning-based methods achieve strong empirical performance but lack theoretical safety guarantees, while control-theoretic methods enforce safety but often lead to overly conservative and inefficient behaviors. We propose a hierarchical multi-agent reinforcement learning framework that enforces hard safety constraints under mild assumptions at low level via a constraint manifold, while enabling effective coordination through high-level policy learning. Our approach provides theoretical safety guarantees in the multi-agent setting and yields stationary learning dynamics, thereby enabling stable and efficient training. Empirically, our method achieves competitive performance while maintaining nearly perfect safety rates, and generalizes effectively to varying numbers of agents and obstacles.

Problem

• multi-agent systems
• safety constraints
• reinforcement learning
• coordination
• generalization

Innovation

• hierarchical multi-agent reinforcement learning
• constraint manifold
• theoretical safety guarantees
• stationary learning dynamics
• safe coordination