Locality-aware Surrogates for Gradient-based Black-box Optimization

2025-01-31

AI Summary

This work addresses black-box simulation optimization problems in physics and engineering where objective functions are non-differentiable. We propose a locally aware surrogate model grounded in the gradient theorem. Our key contribution is the first theoretical linkage between gradient alignment and minimization of the Gradient Path Integral Equation (GradPIE) loss, enabling reliable local gradient modeling without access to true gradients by enforcing local gradient consistency via GradPIE loss regularization. The method integrates active surrogate modeling with black-box query-driven joint online/offline training. Evaluated on three real-world tasks—coupled nonlinear oscillators, analog circuit design, and optical system optimization—the approach achieves significantly improved optimization efficiency under limited query budgets, demonstrating faster and more stable convergence compared to state-of-the-art baselines.

Technology Category

• Search and Optimization: Sampling/Simulation-based Search
• Machine Learning: Optimization
• Reasoning under Uncertainty: Stochastic Optimization

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Graph Algorithms and Modeling for the Web: Graph embeddings and representation learning for Web-related graphs

Abstract

In physics and engineering, many processes are modeled using non-differentiable black-box simulators, making the optimization of such functions particularly challenging. To address such cases, inspired by the Gradient Theorem, we propose locality-aware surrogate models for active model-based black-box optimization. We first establish a theoretical connection between gradient alignment and the minimization of a Gradient Path Integral Equation (GradPIE) loss, which enforces consistency of the surrogate's gradients in local regions of the design space. Leveraging this theoretical insight, we develop a scalable training algorithm that minimizes the GradPIE loss, enabling both offline and online learning while maintaining computational efficiency. We evaluate our approach on three real-world tasks - spanning automated in silico experiments such as coupled nonlinear oscillators, analog circuits, and optical systems - and demonstrate consistent improvements in optimization efficiency under limited query budgets. Our results offer dependable solutions for both offline and online optimization tasks where reliable gradient estimation is needed.

Problem

• Black-box Optimization
• Complex Model
• Physics and Engineering

Innovation

• Position-aware Optimization
• Enhanced Accuracy
• Resource-efficient Performance