Adaptive Stochastic Natural Gradient Method for Safe Optimization on Binary Space

2026-05-18

AI Summary

This work addresses the challenge of avoiding unsafe evaluations in binary-space optimization within high-stakes domains such as healthcare and engineering. It introduces, for the first time, a safety-aware mechanism into the Adaptive Stochastic Natural Gradient (ASNG) framework by leveraging a Walsh-function-based surrogate model. By estimating the Lipschitz constant of this surrogate with respect to Hamming distance, the method delineates a safe region and projects newly generated candidate solutions onto the nearest point within this region, thereby effectively preventing unsafe evaluations. Empirical results on standard binary benchmark problems demonstrate that the proposed approach significantly reduces the number of unsafe solution evaluations while maintaining competitive optimization performance, outperforming existing methods that lack effective risk constraints.

Abstract

Optimization problems in real-world applications across the medical and engineering domains often involve potential risks when evaluating candidate solutions. Safe optimization aims to perform optimization while suppressing unsafe solution evaluations in such situations. For continuous search spaces, there exist safe optimization methods based on evolutionary computation. However, the algorithm development of safe optimization methods for binary search spaces has not been adequately addressed. In this study, we incorporate additional mechanisms for safe optimization into a binary optimization method, the adaptive stochastic natural gradient method (ASNG) with a family of Bernoulli distributions. For safety functions that must be kept non-negative during optimization, the proposed method, safe ASNG, estimates the Lipschitz constants with respect to the Hamming distance by constructing surrogate models of safety functions based on discrete Walsh functions. Then, safe ASNG computes a safe region that consists of safe solutions around the previously evaluated safe solutions. By projecting newly generated solutions to their nearest neighbors within the safe region, safe ASNG suppresses unsafe solution evaluations. Experimental results on benchmark problems on binary domains confirm that, while the comparative methods fail to suppress unsafe solution evaluations, safe ASNG achieves efficient optimization while effectively suppressing unsafe solution evaluations.

Problem

• safe optimization
• binary space
• stochastic natural gradient
• Lipschitz constant
• Hamming distance

Innovation

• safe optimization
• binary space
• stochastic natural gradient
• Lipschitz constant
• Walsh functions