Expected Improvement via Gradient Norms

2026-01-29

AI Summary

This work addresses the tendency of the Expected Improvement (EI) acquisition function in Bayesian optimization to overemphasize exploitation and become trapped in suboptimal stationary points. To mitigate this limitation, the authors propose a novel acquisition function, EI-GN, which incorporates the gradient norm into an enhanced improvement framework for the first time. By defining improvement with respect to a gradient-aware auxiliary objective, EI-GN guides sampling toward regions that balance high reward and proximity to first-order stationarity. A gradient-augmented Gaussian process surrogate model is constructed from function observations, enabling closed-form gradient inference. While preserving the foundational structure of classical EI, EI-GN actively explores promising regions near stationary points. Empirical evaluations demonstrate consistent superiority over existing baselines on standard Bayesian optimization benchmarks and successful application to policy learning tasks in control.

Technology Category

• Search and Optimization: Learning to Search
• Machine Learning: Optimization
• Reasoning under Uncertainty: Stochastic Optimization

Application Category

• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Economics, Online Markets and Human Computation: Social networks and social learning

Abstract

Bayesian Optimization (BO) is a principled approach for optimizing expensive black-box functions, with Expected Improvement (EI) being one of the most widely used acquisition functions. Despite its empirical success, EI is known to be overly exploitative and can converge to suboptimal stationary points. We propose Expected Improvement via Gradient Norms (EI-GN), a novel acquisition function that applies the improvement principle to a gradient-aware auxiliary objective, thereby promoting sampling in regions that are both high-performing and approaching first-order stationarity. EI-GN relies on gradient observations used to learn gradient-enhanced surrogate models that enable principled gradient inference from function evaluations. We derive a tractable closed-form expression for EI-GN that allows efficient optimization and show that the proposed acquisition is consistent with the improvement-based acquisition framework. Empirical evaluations on standard BO benchmarks demonstrate that EI-GN yields consistent improvements against standard baselines. We further demonstrate applicability of EI-GN to control policy learning problems.

Problem

• Bayesian Optimization
• Expected Improvement
• Gradient Norms
• Acquisition Function
• Black-box Optimization

Innovation

• Expected Improvement
• Gradient Norms
• Bayesian Optimization
• Gradient-enhanced Surrogate Models
• Acquisition Function