Bridging Spherical Black-Box Optimizers

2026-06-24

AI Summary

This work addresses the lack of a unified theoretical framework in gradient-free black-box optimization, where existing methods struggle to simultaneously achieve strong performance, robustness, and multimodal optimization capability. Under a spherical symmetry assumption, the authors develop a unified modeling framework that systematically reveals the shared structures and fundamental differences among evolutionary strategies (ES), consensus-based optimization (CBO), and optimization via integration (OVI). Building on this insight, they propose a plug-and-play hybrid optimizer that enables smooth interpolation between algorithms by modulating the fitness aggregation mechanism and consensus radius. The resulting ES-OVI and CBO-OVI variants favor flat minima while balancing high-dimensional efficiency and multimodal exploration. Empirical evaluations on standard benchmarks, high-dimensional motion control, and language model alignment tasks demonstrate consistent superiority over baseline methods, particularly under limited evaluation budgets.

Abstract

When gradient information is unavailable, black-box optimization (BBO) methods provide a practical alternative. While Evolution Strategies (ES), Consensus-Based Optimization (CBO), Optimization via Integration (OVI), and related methods have each been studied independently, their connections remain underexplored. We unify these approaches within a common theoretical framework, revealing that they differ primarily in two design choices: fitness aggregation (controlling sharpness preference) and consensus scope (controlling modality). Leveraging these insights, we introduce hybrid optimizers that interpolate between existing methods. Our ES-OVI hybrid allows explicit control over the preference for flat minima, enabling a trade-off between performance and robustness in continuous control tasks. Our CBO-OVI hybrids combine the higher-dimensional efficiency of parametric methods with the multimodal capabilities of particle-based approaches, achieving competitive results on language model merging under limited evaluation budgets. We validate our methods on standard BBO benchmarks and higher-dimensional locomotion tasks, demonstrating that the hybrid methods can outperform their constituent algorithms.

Problem

• black-box optimization
• Evolution Strategies
• Consensus-Based Optimization
• Optimization via Integration
• hybrid optimizers

Innovation

• Black-Box Optimization
• Hybrid Optimizers
• Flat Minima Preference
• Multimodal Optimization
• Gradient-Free Methods