🤖 AI Summary
This work addresses the challenges of high inference cost, deployment difficulty, and opaque decision-making in deep reinforcement learning for power grid topology control. The authors propose a stress-focused data collection strategy to train a Proximal Policy Optimization (PPO) teacher model and, for the first time, distill it into interpretable, lightweight agents—specifically decision trees and random forests—targeting high-load critical states. The distilled models not only surpass the original PPO policy in average reward and survival duration while significantly reducing inference overhead, but also maintain highly consistent action outputs, enabling human auditability. Furthermore, the study reveals fundamental differences in feature dependencies between neural policies and tree-based models, achieving a balanced trade-off among performance, real-time responsiveness, and interpretability.
📝 Abstract
Deep reinforcement learning (RL) offers a promising route to real-time power grid operation, yet large neural policies are costly to evaluate, hard to deploy on constrained hardware, and opaque to operators. We ask whether a Proximal Policy Optimization (PPO) agent for grid topology control can be compressed into compact tree-based surrogates without losing operational performance. A PPO teacher is trained on Grid2Op's standard 14-bus environment with a stability-oriented reward, using stress-focused data collection on critical, high-loading states. The policy is then distilled into a decision tree and a random forest. Across held-out validation episodes, both surrogates exceed the teacher in mean reward and survival length at a fraction of the inference cost. The decision tree shows high exact-action agreement with the PPO argmax and near-complete agreement within its top-ranked actions, while remaining small enough to be inspected directly. Feature-importance analysis reveals a representational shift: the PPO policy relies mainly on line-loading signals, while the distilled tree is driven primarily by bus-topology variables. These results suggest that stress-focused distillation can convert a black-box neural controller into a lightweight, auditable rule-like surrogate suited for real-time deployment, while also surfacing risks tied to deterministic actions and topology-specific generalization.