Uncertainty-Aware Critic Augmentation for Hierarchical Multi-Agent EV Charging Control

2024-12-23

arXiv.org

AI Summary

Coordinating office building energy management with electric vehicle (EV) charging faces challenges including peak power constraints, time-varying electricity pricing, and uncertainty in EV departure times. Method: This paper proposes HUCA—a real-time hierarchical multi-agent charging control framework—integrating an uncertainty-aware critic enhancement mechanism that explicitly models stochastic EV departure times within the policy evaluation process for improved robustness and adaptability. It employs a hierarchical Actor-Critic architecture combined with multi-agent reinforcement learning to jointly optimize building electricity cost and EV charging completion rate. Results: Experiments using real-world electricity price data demonstrate that HUCA significantly reduces total electricity expenditure while maintaining a high charging satisfaction rate, thereby supporting grid stability and enabling responsive demand-side participation in emergency scenarios.

Technology Category

• Humans and AI: Human-Aware Planning and Behavior Prediction
• Planning, Routing, and Scheduling: Scheduling under Uncertainty
• Multiagent Systems: Mechanism Design

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Energy management for devices in mobile Web and WoT environments
• Economics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystems
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

The advanced bidirectional EV charging and discharging technology, aimed at supporting grid stability and emergency operations, has driven a growing interest in workplace applications. It not only reduces electricity expenses but also enhances the resilience in handling practical matters, such as peak power limitation, fluctuating energy prices, and unpredictable EV departures. Considering these factors systematically can benefit energy efficiency in office buildings and for EV users simultaneously. To employ AI to address these issues, we propose HUCA, a novel real-time charging control for regulating energy demands for both the building and EVs. HUCA employs hierarchical actor-critic networks to dynamically reduce electricity costs in buildings, accounting for the needs of EV charging in the dynamic pricing scenario. To tackle the uncertain EV departures, we introduce a new critic augmentation to account for departure uncertainties in evaluating the charging decisions, while maintaining the robustness of the charging control. Experiments on real-world electricity datasets under both simulated certain and uncertain departure scenarios demonstrate that HUCA outperforms baselines in terms of total electricity costs while maintaining competitive performance in fulfilling EV charging requirements. A case study also manifests that HUCA effectively balances energy supply between the building and EVs based on real-time information, showcasing its potential as a key AI-driven solution for vehicle charging control.

Problem

• Hierarchical Multi-Agent EV Charging Control
• Dynamic Electricity Cost Reduction
• Uncertain EV Departure Management

Innovation

• Hierarchical actor-critic networks used
• Critic augmentation for uncertainty handling
• Real-time EV charging control implemented