A Generative Model Enhanced Multi-Agent Reinforcement Learning Method for Electric Vehicle Charging Navigation

2025-02-27

AI Summary

Real-time navigation for electric vehicle (EV) charging is challenged by dynamic traffic conditions, time-varying electricity pricing, and stochastic charging station competition. Method: This paper proposes a fully decentralized multi-agent deep reinforcement learning framework relying solely on local observations. It introduces a CVAE-LSTM recommendation model and a future charging competition encoder to compress global system states into lightweight, agent-specific representations; further, it employs the Multi-Gradient Descent Algorithm (MGDA) to enable stable, bi-objective training—jointly optimizing charging cost and waiting time. Results: Evaluated on a realistic urban-scale simulation of Xi’an, the approach achieves near-optimal performance—within 8% of the centralized oracle—without any global communication. It significantly reduces communication overhead and eliminates privacy leakage risks associated with centralized data sharing, establishing a scalable, privacy-preserving paradigm for distributed EV charging navigation.

Technology Category

• Planning, Routing, and Scheduling: Planning with Language Models
• Multiagent Systems: Adversarial Agents
• Search and Optimization: Learning to Search

Application Category

• User Modeling, Personalization and Recommendation: User privacy protection in personalized systems
• Economics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystems
• Systems and Infrastructure for Web, Mobile and WoT: Energy management for devices in mobile Web and WoT environments

Abstract

With the widespread adoption of electric vehicles (EVs), navigating for EV drivers to select a cost-effective charging station has become an important yet challenging issue due to dynamic traffic conditions, fluctuating electricity prices, and potential competition from other EVs. The state-of-the-art deep reinforcement learning (DRL) algorithms for solving this task still require global information about all EVs at the execution stage, which not only increases communication costs but also raises privacy issues among EV drivers. To overcome these drawbacks, we introduce a novel generative model-enhanced multi-agent DRL algorithm that utilizes only the EV's local information while achieving performance comparable to these state-of-the-art algorithms. Specifically, the policy network is implemented on the EV side, and a Conditional Variational Autoencoder-Long Short Term Memory (CVAE-LSTM)-based recommendation model is developed to provide recommendation information. Furthermore, a novel future charging competition encoder is designed to effectively compress global information, enhancing training performance. The multi-gradient descent algorithm (MGDA) is also utilized to adaptively balance the weight between the two parts of the training objective, resulting in a more stable training process. Simulations are conducted based on a practical area in Xi'an, China. Experimental results show that our proposed algorithm, which relies on local information, outperforms existing local information-based methods and achieves less than 8% performance loss compared to global information-based methods.

Problem

• EV charging navigation optimization
• Privacy-preserving local information utilization
• Enhanced multi-agent reinforcement learning

Innovation

• Generative model-enhanced DRL
• CVAE-LSTM recommendation model
• Future charging competition encoder