Self-Certifying Primal-Dual Optimization Proxies for Large-Scale Batch Economic Dispatch

2025-10-17

AI Summary

Deploying trustworthy optimization agents for large-scale bulk economic dispatch remains challenging due to stringent requirements on solution reliability and computational efficiency. Method: This paper proposes a self-verifying primal-dual learning-based optimization agent that synergistically integrates classical solvers with data-driven surrogates. It introduces a novel duality-theoretic self-certification mechanism to quantify the optimality gap of predicted solutions in real time, coupled with a primal-dual joint training framework and a user-defined threshold-triggered fallback strategy—reverting to parallel simplex when needed. Contribution/Results: On large-scale transmission systems, the method achieves over 1,000× speedup relative to conventional solvers while rigorously guaranteeing an optimality gap ≤2%. This significantly improves the interpretability and engineering reliability of the speed–optimality trade-off, enabling trustworthy deployment in practical power system operations.

Technology Category

• Search and Optimization: Learning to Search
• Reasoning under Uncertainty: Stochastic Optimization
• Constraint Satisfaction and Optimization: Constraint Learning and Acquisition

Application Category

• Economics, Online Markets and Human Computation: Trust and reliance of crowd workers and data experts on GenAI
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Security and Privacy: Large-scale security measurements

Abstract

Recent research has shown that optimization proxies can be trained to high fidelity, achieving average optimality gaps under 1% for large-scale problems. However, worst-case analyses show that there exist in-distribution queries that result in orders of magnitude higher optimality gap, making it difficult to trust the predictions in practice. This paper aims at striking a balance between classical solvers and optimization proxies in order to enable trustworthy deployments with interpretable speed-optimality tradeoffs based on a user-defined optimality threshold. To this end, the paper proposes a hybrid solver that leverages duality theory to efficiently bound the optimality gap of predictions, falling back to a classical solver for queries where optimality cannot be certified. To improve the achieved speedup of the hybrid solver, the paper proposes an alternative training procedure that combines the primal and dual proxy training. Experiments on large-scale transmission systems show that the hybrid solver is highly scalable. The proposed hybrid solver achieves speedups of over 1000x compared to a parallelized simplex-based solver while guaranteeing a maximum optimality gap of 2%.

Problem

• Ensuring trustworthy optimization proxies by certifying prediction optimality gaps
• Balancing computational speed and solution quality in economic dispatch
• Developing hybrid solver with primal-dual training for large-scale power systems

Innovation

• Hybrid solver combines optimization proxies with classical methods
• Uses duality theory to certify optimality gap bounds
• Joint primal-dual proxy training enhances computational speedup