Discovering Quality-Diversity Algorithms via Meta-Black-Box Optimization

2025-02-04

AI Summary

This work addresses the poor generalizability and trade-off between diversity and performance in quality diversity (QD) algorithms—stemming from their reliance on hand-crafted heuristics (e.g., grid- or neighborhood-based competition). We propose the first automated QD algorithm discovery framework, integrating meta black-box optimization with attention-based neural networks. Methodologically, we formulate QD as a learnable, attention-driven policy that implicitly models population dynamics in descriptor space, trained solely on fitness signals without explicit diversity objectives. Key contributions include: (i) the first end-to-end neural architecture search for QD algorithms; (ii) emergent, self-sustained high behavioral diversity, revealing diversity as an intrinsic, emergent property of optimization; and (iii) superior performance over MAP-Elites and other baselines on multi-task benchmarks and out-of-distribution robotic control tasks—while generalizing to high-dimensional descriptors and large populations.

Technology Category

• Search and Optimization: Learning to Search
• Intelligent Robots: Learning & Optimization for ROB
• Humans and AI: Human-Aware Planning and Behavior Prediction

Application Category

• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work
• Web Mining and Content Analysis: Web data quality in the era of algorithmically-generated content
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web

Abstract

Quality-Diversity has emerged as a powerful family of evolutionary algorithms that generate diverse populations of high-performing solutions by implementing local competition principles inspired by biological evolution. While these algorithms successfully foster diversity and innovation, their specific mechanisms rely on heuristics, such as grid-based competition in MAP-Elites or nearest-neighbor competition in unstructured archives. In this work, we propose a fundamentally different approach: using meta-learning to automatically discover novel Quality-Diversity algorithms. By parameterizing the competition rules using attention-based neural architectures, we evolve new algorithms that capture complex relationships between individuals in the descriptor space. Our discovered algorithms demonstrate competitive or superior performance compared to established Quality-Diversity baselines while exhibiting strong generalization to higher dimensions, larger populations, and out-of-distribution domains like robot control. Notably, even when optimized solely for fitness, these algorithms naturally maintain diverse populations, suggesting meta-learning rediscovers that diversity is fundamental to effective optimization.

Problem

• Automatically discover Quality-Diversity algorithms
• Parameterize competition rules using neural architectures
• Maintain diverse populations via meta-learning optimization

Innovation

• Meta-learning for algorithm discovery
• Attention-based neural architectures
• Evolved Quality-Diversity algorithms