Neural Exploratory Landscape Analysis for Meta-Black-Box-Optimization

2024-08-20

AI Summary

Existing MetaBBO methods rely on handcrafted exploratory landscape analysis (ELA) features, limiting their autonomy and generalization capability. This paper proposes NeurELA—the first end-to-end learnable neural landscape analysis framework that dynamically and adaptively extracts exploration features during black-box optimization. Methodologically, NeurELA fully neuralizes ELA and integrates it into the meta-optimization pipeline; introduces a two-stage attention mechanism to jointly model the coupling between optimization trajectories and problem structure; and adopts a multi-task neural evolutionary pretraining paradigm to enhance representation robustness. Empirical results demonstrate that NeurELA consistently outperforms prior approaches across diverse and unseen MetaBBO tasks, enables efficient fine-tuning, and significantly improves optimizer autonomy—i.e., its capacity for self-aware exploration—and cross-task generalization performance.

Technology Category

• Search and Optimization: Metareasoning and Metaheuristics
• Intelligent Robots: Learning & Optimization for ROB
• Machine Learning: Optimization

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Recent research in Meta-Black-Box Optimization (MetaBBO) have shown that meta-trained neural networks can effectively guide the design of black-box optimizers, significantly reducing the need for expert tuning and delivering robust performance across complex problem distributions. Despite their success, a paradox remains: MetaBBO still rely on human-crafted Exploratory Landscape Analysis features to inform the meta-level agent about the low-level optimization progress. To address the gap, this paper proposes Neural Exploratory Landscape Analysis (NeurELA), a novel framework that dynamically profiles landscape features through a two-stage, attention-based neural network, executed in an entirely end-to-end fashion. NeurELA is pre-trained over a variety of MetaBBO algorithms using a multi-task neuroevolution strategy. Extensive experiments show that NeurELA achieves consistently superior performance when integrated into different and even unseen MetaBBO tasks and can be efficiently fine-tuned for further performance boost. This advancement marks a pivotal step in making MetaBBO algorithms more autonomous and broadly applicable. The source code of NeurELA can be accessed at https://github.com/GMC-DRL/Neur-ELA.

Problem

• Replacing human-crafted features with neural networks for optimization feedback
• Enhancing MetaBBO autonomy via end-to-end neural landscape analysis
• Improving performance across diverse and unseen optimization tasks

Innovation

• Neural Exploratory Landscape Analysis (NeurELA) framework
• Two-stage attention-based neural network
• Multi-task neuroevolution pre-training strategy