Minimisation of Quasar-Convex Functions Using Random Zeroth-Order Oracles

2025-05-04

AI Summary

This work investigates the theoretical performance of randomized Gaussian-smoothed zeroth-order (ZO) optimization algorithms for minimizing quasiconvex (QC) and strongly quasiconvex (SQC) functions, under both unconstrained and constrained settings. To address constrained optimization, the authors introduce the novel notion of *proximal quasiconvexity*, enabling rigorous convergence guarantees and neighborhood accuracy control for ZO methods. They establish global convergence and derive tight oracle complexity bounds—matching lower bounds in key regimes—and uncover an intrinsic mechanism by which ZO methods outperform first-order gradient descent in certain nonsmooth, black-box scenarios. Methodologically, the approach integrates Gaussian smoothing for gradient estimation, proximal operator analysis, and variance reduction techniques. Empirical evaluation across diverse machine learning tasks—including robust optimization and adversarial training—demonstrates consistent and significant improvements over standard gradient-based methods.

Technology Category

• Search and Optimization: Non-convex Optimization
• Reasoning under Uncertainty: Stochastic Optimization
• Machine Learning: Optimization

Application Category

• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Security and Privacy: Data transparency and provenance
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods

Abstract

This study explores the performance of a random Gaussian smoothing zeroth-order (ZO) scheme for minimising quasar-convex (QC) and strongly quasar-convex (SQC) functions in both unconstrained and constrained settings. For the unconstrained problem, we establish the ZO algorithm's convergence to a global minimum along with its complexity when applied to both QC and SQC functions. For the constrained problem, we introduce the new notion of proximal-quasar-convexity and prove analogous results to the unconstrained case. Specifically, we show the complexity bounds and the convergence of the algorithm to a neighbourhood of a global minimum whose size can be controlled under a variance reduction scheme. Theoretical findings are illustrated through investigating the performance of the algorithm applied to a range of problems in machine learning and optimisation. Specifically, we observe scenarios where the ZO method outperforms gradient descent. We provide a possible explanation for this phenomenon.

Problem

• Minimizing quasar-convex functions using zeroth-order methods
• Analyzing convergence and complexity in unconstrained and constrained settings
• Comparing zeroth-order performance with gradient descent in machine learning

Innovation

• Random Gaussian smoothing zeroth-order scheme
• Proximal-quasar-convexity for constrained problems
• Variance reduction controls convergence neighborhood