Warm Starting of CMA-ES for Contextual Optimization Problems

2025-02-18

Parallel Problem Solving from Nature

AI Summary

Existing methods for context-dependent black-box optimization struggle to effectively exploit contextual information, and evolutionary algorithms—particularly CMA-ES—lack context-aware initialization strategies. To address this, we propose a context-aware warm-start mechanism tailored for CMA-ES. Our core contribution is the first joint modeling of CMA-ES distribution parameters (mean vector and covariance matrix) with context encodings via a meta-learned parameter mapping network. This enables cross-task compression of distribution states and context-embedding-guided covariance calibration. The resulting initialization is context-driven and transferable across tasks. Evaluated on multi-task benchmarks, our method accelerates convergence by 2.3–5.1× compared to standard CMA-ES and outperforms state-of-the-art meta-optimization approaches, demonstrating superior generalization capability.

Technology Category

• Search and Optimization: Metareasoning and Metaheuristics
• Machine Learning: Evolutionary Learning
• Constraint Satisfaction and Optimization: Distributed CSP/Optimization

Application Category

• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Economics, Online Markets and Human Computation: Trust and reliance of crowd workers and data experts on GenAI

Problem

• Enhancing contextual optimization with warm starting
• Improving CMA-ES for contextual vector adaptation
• Utilizing Gaussian process for better initialization

Innovation

• CMA-ES with contextual warm starting
• Multivariate Gaussian process regression
• Initializes search distribution using posterior