Optimization Proxies using Limited Labeled Data and Training Time - A Semi-Supervised Bayesian Neural Network Approach

2024-10-04

arXiv.org

AI Summary

In engineering-constrained optimization (e.g., energy network scheduling), conventional deep neural network (DNN) surrogates suffer from poor performance and lack of uncertainty quantification under scarce labeled data and tight training time constraints. Method: This paper proposes a “sandwich-style” semi-supervised Bayesian neural network (BNN) surrogate learning framework. It alternates between supervised training—minimizing prediction cost—and unsupervised regularization—enforcing constraint feasibility—integrating variational inference with posterior sampling to yield probabilistic confidence bounds with minimal validation data. Contribution/Results: Evaluated on non-convex energy network optimization, the framework reduces equality constraint violations by up to one order of magnitude and cuts inequality constraint violations in half, significantly advancing state-of-the-art surrogates in both reliability and interpretability under data-scarce, time-critical conditions.

Technology Category

• Reasoning under Uncertainty: Stochastic Optimization
• Constraint Satisfaction and Optimization: Constraint Learning and Acquisition
• Search and Optimization: Sampling/Simulation-based Search

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Systems and Infrastructure for Web, Mobile and WoT: Energy management for devices in mobile Web and WoT environments

Abstract

Constrained optimization problems arise in various engineering systems such as inventory management and power grids. Standard deep neural network (DNN) based machine learning proxies are ineffective in practical settings where labeled data is scarce and training times are limited. We propose a semi-supervised Bayesian Neural Networks (BNNs) based optimization proxy for this complex regime, wherein training commences in a sandwiched fashion, alternating between a supervised learning step for minimizing cost, and an unsupervised learning step for enforcing constraint feasibility. We show that the proposed semi-supervised BNN outperforms DNN architectures on important non-convex constrained optimization problems from energy network operations, achieving up to a tenfold reduction in expected maximum equality gap and halving the inequality gaps. Further, the BNN's ability to provide posterior samples is leveraged to construct practically meaningful probabilistic confidence bounds on performance using a limited validation data, unlike prior methods.

Problem

• Optimization proxies for engineering systems
• Handling scarce labeled data efficiently
• Reducing training time with semi-supervised BNNs

Innovation

• Semi-supervised Bayesian Neural Networks
• Sandwiched training approach
• Probabilistic confidence bounds