Deep Reinforcement Learning-Based Optimization of Second-Life Battery Utilization in Electric Vehicles Charging Stations

2025-02-05

AI Summary

Retired electric vehicle (EV) batteries deployed in second-life battery (SLB) systems at EV charging stations (EVCSs) face dual challenges—stochastic EV arrival/departure patterns and dynamic electricity pricing—compromising operational efficiency and battery longevity. Method: This paper proposes a deep reinforcement learning (DRL)-based operational planning framework, introducing the Soft Actor-Critic (SAC) algorithm for annual-scale offline training and real-time scheduling of SLB-integrated EVCSs. A customized reward function jointly optimizes operational cost, grid load balancing, and battery lifetime constraints. Battery energy storage system (BESS) dynamics and stochastic uncertainty modeling are incorporated to ensure policy convergence and robust decision-making. Results: Experiments demonstrate an average policy convergence rate of 92.7%, a 15.3% reduction in peak-to-valley load difference, and significant reductions in lifecycle planning and operational costs. The framework establishes a scalable, highly adaptable intelligent scheduling paradigm for retired battery repurposing.

Technology Category

• Planning, Routing, and Scheduling: Planning/Scheduling and Learning
• Intelligent Robots: Learning & Optimization for ROB
• Reasoning under Uncertainty: Stochastic Optimization

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Energy management for devices in mobile Web and WoT environments
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

The rapid rise in electric vehicle (EV) adoption presents significant challenges in managing the vast number of retired EV batteries. Research indicates that second-life batteries (SLBs) from EVs typically retain considerable residual capacity, offering extended utility. These batteries can be effectively repurposed for use in EV charging stations (EVCS), providing a cost-effective alternative to new batteries and reducing overall planning costs. Integrating battery energy storage systems (BESS) with SLBs into EVCS is a promising strategy to alleviate system overload. However, efficient operation of EVCS with integrated BESS is hindered by uncertainties such as fluctuating EV arrival and departure times and variable power prices from the grid. This paper presents a deep reinforcement learning-based (DRL) planning framework for EV charging stations with BESS, leveraging SLBs. We employ the advanced soft actor-critic (SAC) approach, training the model on a year's worth of data to account for seasonal variations, including weekdays and holidays. A tailored reward function enables effective offline training, allowing real-time optimization of EVCS operations under uncertainty.

Problem

• Optimize second-life battery use
• Manage EV charging station uncertainties
• Integrate BESS with deep reinforcement learning

Innovation

• Deep reinforcement learning optimizes battery use
• Soft actor-critic method handles seasonal variations
• Tailored reward function facilitates real-time optimization