Optimizing Electric Vehicle Charging Station Placement Using Reinforcement Learning and Agent-Based Simulations

2025-11-02

AI Summary

This paper addresses the dynamic charging station layout optimization problem for electric vehicles, characterized by spatiotemporal uncertainties in traffic flow, user behavior, and charging demand. Method: We propose an adaptive framework integrating deep reinforcement learning (DRL) with agent-based simulation (ABS). A double Q-network architecture and a hybrid reward function are designed; ABS is employed to simulate vehicle routing and charging behaviors in real time, enabling joint optimization of demand forecasting, station siting, and port allocation. Contribution/Results: By embedding simulation feedback into the DRL closed loop, our approach overcomes the limitations of conventional static planning. Empirical evaluation in Hanoi, Vietnam, demonstrates a 53.28% reduction in average waiting time compared to static baselines, confirming the method’s effectiveness, robustness, and scalability under realistic urban conditions.

Technology Category

• Planning, Routing, and Scheduling: Optimization of Spatio-temporal Systems
• Search and Optimization: Sampling/Simulation-based Search
• Reasoning under Uncertainty: Stochastic Optimization

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Systems and Infrastructure for Web, Mobile and WoT: Energy management for devices in mobile Web and WoT environments
• Economics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystems

Abstract

The rapid growth of electric vehicles (EVs) necessitates the strategic placement of charging stations to optimize resource utilization and minimize user inconvenience. Reinforcement learning (RL) offers an innovative approach to identifying optimal charging station locations; however, existing methods face challenges due to their deterministic reward systems, which limit efficiency. Because real-world conditions are dynamic and uncertain, a deterministic reward structure cannot fully capture the complexities of charging station placement. As a result, evaluation becomes costly and time-consuming, and less reflective of real-world scenarios. To address this challenge, we propose a novel framework that integrates deep RL with agent-based simulations to model EV movement and estimate charging demand in real time. Our approach employs a hybrid RL agent with dual Q-networks to select optimal locations and configure charging ports, guided by a hybrid reward function that combines deterministic factors with simulation-derived feedback. Case studies in Hanoi, Vietnam, show that our method reduces average waiting times by 53.28% compared to the initial state, outperforming static baseline methods. This scalable and adaptive solution enhances EV infrastructure planning, effectively addressing real-world complexities and improving user experience.

Problem

• Optimizing EV charging station placement using reinforcement learning
• Addressing dynamic real-world conditions with agent-based simulations
• Reducing waiting times and improving charging infrastructure efficiency

Innovation

• Integrates deep reinforcement learning with agent-based simulations
• Uses dual Q-networks for location selection and port configuration
• Employs hybrid reward function combining deterministic and simulation feedback