Hierarchical Multi-Agent Reinforcement Learning with Control Barrier Functions for Safety-Critical Autonomous Systems

2025-07-20

AI Summary

In safety-critical multi-agent systems, jointly optimizing collaborative task performance and individual agent safety remains challenging. Method: This paper proposes a skill-abstraction-based hierarchical reinforcement learning framework: a high-level policy coordinates multi-agent collaboration, while a low-level controller enforces single-agent motion safety via Control Barrier Functions (CBFs), achieving the first decoupling and tight integration of cooperation and safety at the policy learning level. The approach unifies hierarchical multi-agent RL, CBF-constrained policy embedding, skill-driven behavioral abstraction, and constrained optimization. Results: Evaluated in dense, dynamic road scenarios, the method achieves a safety success rate exceeding 95%, outperforming state-of-the-art baselines significantly, while maintaining high task completion rates and strong safety robustness.

Technology Category

• Multiagent Systems: Multiagent Learning
• Humans and AI: Human-Aware Planning and Behavior Prediction
• Natural Language Processing: Safety and Robustness

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Search and Retrieval-Augmented AI: Agentic search
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

We address the problem of safe policy learning in multi-agent safety-critical autonomous systems. In such systems, it is necessary for each agent to meet the safety requirements at all times while also cooperating with other agents to accomplish the task. Toward this end, we propose a safe Hierarchical Multi-Agent Reinforcement Learning (HMARL) approach based on Control Barrier Functions (CBFs). Our proposed hierarchical approach decomposes the overall reinforcement learning problem into two levels learning joint cooperative behavior at the higher level and learning safe individual behavior at the lower or agent level conditioned on the high-level policy. Specifically, we propose a skill-based HMARL-CBF algorithm in which the higher level problem involves learning a joint policy over the skills for all the agents and the lower-level problem involves learning policies to execute the skills safely with CBFs. We validate our approach on challenging environment scenarios whereby a large number of agents have to safely navigate through conflicting road networks. Compared with existing state of the art methods, our approach significantly improves the safety achieving near perfect (within 5%) success/safety rate while also improving performance across all the environments.

Problem

• Safe policy learning in multi-agent autonomous systems
• Ensuring safety while cooperating among agents
• Hierarchical reinforcement learning with barrier functions

Innovation

• Hierarchical Multi-Agent Reinforcement Learning for safety
• Control Barrier Functions ensure individual agent safety
• Skill-based HMARL-CBF algorithm improves success rates