Learning Low-Dimensional Embeddings for Black-Box Optimization

2025-05-02

AI Summary

For high-dimensional black-box optimization problems with inaccessible gradients and limited evaluation budgets, this paper proposes a meta-learning-based approach to construct transferable low-dimensional embedding manifolds. The method operates in two phases: during meta-training, it learns a shared nonlinear dimensionality-reduction manifold from a set of related problem instances; at inference, it rapidly projects a new high-dimensional objective function onto this pre-learned manifold and performs efficient optimization in the low-dimensional space. To our knowledge, this is the first work to apply meta-learning for learning generalizable, problem-class-specific optimization manifolds—enabling cross-instance adaptive dimensionality compression and knowledge transfer. Experiments demonstrate that our method substantially reduces trial-and-error overhead, achieves faster convergence to near-optimal solutions under constrained evaluation budgets, and exhibits both superior optimization efficiency and strong generalization across unseen tasks.

Technology Category

• Machine Learning: Learning with Manifolds
• Search and Optimization: Metareasoning and Metaheuristics
• Computer Vision: Learning & Optimization for CV

Application Category

• Graph Algorithms and Modeling for the Web: Graph embeddings and representation learning for Web-related graphs
• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

When gradient-based methods are impractical, black-box optimization (BBO) provides a valuable alternative. However, BBO often struggles with high-dimensional problems and limited trial budgets. In this work, we propose a novel approach based on meta-learning to pre-compute a reduced-dimensional manifold where optimal points lie for a specific class of optimization problems. When optimizing a new problem instance sampled from the class, black-box optimization is carried out in the reduced-dimensional space, effectively reducing the effort required for finding near-optimal solutions.

Problem

• High-dimensional black-box optimization challenges
• Limited trial budgets in optimization
• Finding optimal points in reduced-dimensional space

Innovation

• Meta-learning for low-dimensional embeddings
• Pre-computing reduced-dimensional manifolds
• Optimizing in reduced space for efficiency