An $(epsilon,delta)$-accurate level set estimation with a stopping criterion

2025-03-26

AI Summary

This work addresses the level-set estimation problem for unknown, expensive-to-evaluate functions—i.e., efficiently identifying the super-level set (region where function values exceed a given threshold) under a limited budget of function evaluations. We propose the first theoretically grounded automatic stopping criterion for this task. Our method employs a Gaussian process surrogate model and introduces a novel acquisition function that integrates statistical confidence bounds within a sequential Bayesian optimization framework. We establish rigorous theoretical guarantees: the algorithm achieves $epsilon$-accuracy with probability at least $1-delta$, and we derive provable lower bounds on performance metrics such as the F-score. Experiments demonstrate that our approach matches the identification accuracy of state-of-the-art methods while substantially reducing redundant evaluations, enabling provably correct, adaptive, and timely termination.

Technology Category

• Reasoning under Uncertainty: Stochastic Optimization
• Machine Learning: Calibration & Uncertainty Quantification
• Search and Optimization: Mixed Discrete/Continuous Search

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

The level set estimation problem seeks to identify regions within a set of candidate points where an unknown and costly to evaluate function's value exceeds a specified threshold, providing an efficient alternative to exhaustive evaluations of function values. Traditional methods often use sequential optimization strategies to find $epsilon$-accurate solutions, which permit a margin around the threshold contour but frequently lack effective stopping criteria, leading to excessive exploration and inefficiencies. This paper introduces an acquisition strategy for level set estimation that incorporates a stopping criterion, ensuring the algorithm halts when further exploration is unlikely to yield improvements, thereby reducing unnecessary function evaluations. We theoretically prove that our method satisfies $epsilon$-accuracy with a confidence level of $1 - delta$, addressing a key gap in existing approaches. Furthermore, we show that this also leads to guarantees on the lower bounds of performance metrics such as F-score. Numerical experiments demonstrate that the proposed acquisition function achieves comparable precision to existing methods while confirming that the stopping criterion effectively terminates the algorithm once adequate exploration is completed.

Problem

• Estimates level sets of costly functions efficiently
• Introduces stopping criterion to prevent over-exploration
• Ensures ε-accuracy with 1-δ confidence theoretically

Innovation

• Stopping criterion for efficient level set estimation
• Theoretical proof of ε-accuracy with confidence
• Guarantees on performance metrics like F-score