Extract-QD Framework: A Generic Approach for Quality-Diversity in Noisy, Stochastic or Uncertain Domains

2025-02-10

AI Summary

Quality Diversity (QD) optimization under noise and uncertainty remains challenging due to fragmented, non-interoperable uncertainty-aware QD (UQD) methods. Method: We propose EQD, the first modular, unified Uncertainty-aware QD framework based on an Extract-QD architecture enabling plug-and-play UQD customization; and EME, a novel algorithm integrating policy-gradient enhancement, solution-set resampling, and uncertainty-robust evaluation. Results: EME is the first UQD algorithm to consistently outperform all existing UQD baselines across standard benchmarks. EQD incurs zero overhead when enhancing PG-MAP-Elites, while EME achieves state-of-the-art or competitive performance across diverse tasks. Our contributions are threefold: (1) the first modular, unified UQD framework; (2) the first UQD algorithm empirically superior to all prior methods on established benchmarks; and (3) a practical design that improves both performance and computational efficiency, significantly lowering the barrier to UQD adoption and evaluation cost.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Search and Optimization: Sampling/Simulation-based Search
• Reasoning under Uncertainty: Stochastic Optimization

Application Category

• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Quality-Diversity (QD) has demonstrated potential in discovering collections of diverse solutions to optimisation problems. Originally designed for deterministic environments, QD has been extended to noisy, stochastic, or uncertain domains through various Uncertain-QD (UQD) methods. However, the large number of UQD methods, each with unique constraints, makes selecting the most suitable one challenging. To remedy this situation, we present two contributions: first, the Extract-QD Framework (EQD Framework), and second, Extract-ME (EME), a new method derived from it. The EQD Framework unifies existing approaches within a modular view, and facilitates developing novel methods by interchanging modules. We use it to derive EME, a novel method that consistently outperforms or matches the best existing methods on standard benchmarks, while previous methods show varying performance. In a second experiment, we show how our EQD Framework can be used to augment existing QD algorithms and in particular the well-established Policy-Gradient-Assisted-MAP-Elites method, and demonstrate improved performance in uncertain domains at no additional evaluation cost. For any new uncertain task, our contributions now provide EME as a reliable"first guess"method, and the EQD Framework as a tool for developing task-specific approaches. Together, these contributions aim to lower the cost of adopting UQD insights in QD applications.

Problem

• Unifies Quality-Diversity methods for uncertain domains.
• Introduces Extract-QD Framework for modular method development.
• Enhances existing QD algorithms in noisy environments.

Innovation

• EQD Framework unifies UQD methods
• EME method outperforms existing benchmarks
• EQD enhances Policy-Gradient-Assisted-MAP-Elites