Sampling-Based Coordination-Informed Multi-Objective Multi-Robot Reinforcement Learning

2026-06-29

AI Summary

This work addresses the challenge of balancing multi-objective optimization and coordination efficiency in distributed multi-robot systems. The authors propose CIMORL, a novel framework that integrates distributed weight prediction, privileged expert training, tree search (TS), and model predictive path integral (MPPI) sampling. During training, global information is leveraged to ensure Pareto optimality, while deployment remains fully decentralized. CIMORL unifies multi-objective decision-making with collective coordination, achieving a 21.2% improvement in hypervolume metric over existing methods in both cooperative and competitive scenarios. The approach demonstrates superior policy stability and robustness under partial observability, as validated on the Crazyflie drone platform.

Abstract

Multi-robot systems must simultaneously optimize competing objectives while maintaining coordinated behavior. Existing multi-agent reinforcement learning approaches often rely on fixed or centralized coordination, which limits adaptability and violates distributed constraints. This work introduces the Coordination-Informed Multi-Objective Reinforcement Learning (CIMORL) framework, integrating a distributed weight prediction mechanism, a privileged expert training strategy, and theoretical guarantees for Pareto-optimal solutions. We present the base CIMORL method alongside two sampling-based variants, CIMORL-TS (Tree Search) and CIMORL-MPPI (MPPI), which leverage privileged global information during training to enable fully decentralized deployment. Experimental validation in cooperative and adversarial scenarios demonstrates a $21.2\%$ hypervolume improvement and superior policy stability compared to state-of-the-art baselines. Real-world experiments with Crazyflie drones further validate the framework's robustness in resource allocation and multi-attacker multi-defend scenarios under partial observability.

Problem

• multi-robot systems
• multi-objective optimization
• coordination
• distributed constraints
• reinforcement learning

Innovation

• multi-objective reinforcement learning
• distributed coordination
• Pareto optimality
• sampling-based planning
• decentralized multi-robot systems