Multi-Agent Soft Actor-Critic with Global Loss for Autonomous Mobility-on-Demand Fleet Control

2024-04-10

arXiv.org

AI Summary

This work addresses the profit maximization problem for centralized operators in Autonomous Mobility-on-Demand (AMoD) systems, focusing on the sequential joint decision-making of vehicle dispatching and dynamic rebalancing. We propose a vehicle-centric multi-agent reinforcement learning framework, introducing a novel global-action-aware critic loss function. Furthermore, we pioneer the integration of Soft Actor-Critic (SAC) with weighted bipartite graph matching to jointly optimize request-vehicle assignment and coordinated rebalancing. The proposed method improves dispatching performance by 12.9% over the current state-of-the-art (SOTA). When combined with the integrated rebalancing strategy, overall operational profit increases by 38.9%. These gains significantly enhance both the economic viability and scalability of AMoD systems.

Technology Category

• Planning, Routing, and Scheduling: Learning for Planning and Scheduling
• Search and Optimization: Learning to Search
• Multiagent Systems: Multiagent Learning

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs

Abstract

We study a sequential decision-making problem for a profit-maximizing operator of an Autonomous Mobility-on-Demand system. Optimizing a central operator's vehicle-to-request dispatching policy requires efficient and effective fleet control strategies. To this end, we employ a multi-agent Soft Actor-Critic algorithm combined with weighted bipartite matching. We propose a novel vehicle-based algorithm architecture and adapt the critic's loss function to appropriately consider global actions. Furthermore, we extend our algorithm to incorporate rebalancing capabilities. Through numerical experiments, we show that our approach outperforms state-of-the-art benchmarks by up to 12.9% for dispatching and up to 38.9% with integrated rebalancing.

Problem

• Optimize vehicle dispatching for autonomous mobility systems
• Develop efficient multi-agent fleet control strategies
• Enhance performance with coordinated actions and rebalancing

Innovation

• Multi-agent Soft Actor-Critic algorithm
• Weighted bipartite matching integration
• Vehicle-based architecture with coordinated loss