Exploring the Performance-Reproducibility Trade-off in Quality-Diversity

2024-09-20

arXiv.org

AI Summary

This work addresses the inherent trade-off between performance and reproducibility in Quality-Diversity (QD) optimization under uncertainty—a conflict previously unformalized. Existing QD methods often neglect this tension, resulting in poor simultaneous attainment of high adaptability and stability in uncertain domains (e.g., robust legged locomotion). To resolve this, we propose the first QD framework explicitly modeling the performance–reproducibility trade-off: an extension of MAP-Elites integrating multi-objective optimization, uncertainty-aware modeling, and preference embedding. We design four prior-driven and one posterior-driven algorithms, accommodating both known- and unknown-preference scenarios. Evaluated on multiple uncertain benchmark tasks, our approach significantly outperforms state-of-the-art uncertain QD methods. Crucially, it uncovers and activates critical regions of the solution space—diverse, reproducible, and high-performing—that had been overlooked by conventional single-objective (performance-only) QD paradigms.

Technology Category

• Reasoning under Uncertainty: Stochastic Optimization
• Search and Optimization: Sampling/Simulation-based Search
• Machine Learning: Calibration & Uncertainty Quantification

Application Category

• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

Quality-Diversity (QD) algorithms have exhibited promising results across many domains and applications. However, uncertainty in fitness and behaviour estimations of solutions remains a major challenge when QD is used in complex real-world applications. While several approaches have been proposed to improve the performance in uncertain applications, many fail to address a key challenge: determining how to prioritise solutions that perform consistently under uncertainty, in other words, solutions that are reproducible. Most prior methods improve fitness and reproducibility jointly, ignoring the possibility that they could be contradictory objectives. For example, in robotics, solutions may reliably walk at 90% of the maximum velocity in uncertain environments, while solutions that walk faster are also more prone to falling over. As this is a trade-off, neither one of these two solutions is"better"than the other. Thus, algorithms cannot intrinsically select one solution over the other, but can only enforce given preferences over these two contradictory objectives. In this paper, we formalise this problem as the performance-reproducibility trade-off for uncertain QD. We propose four new a-priori QD algorithms that find optimal solutions for given preferences over the trade-offs. We also propose an a-posteriori QD algorithm for when these preferences cannot be defined in advance. Our results show that our approaches successfully find solutions that satisfy given preferences. Importantly, by simply accounting for this trade-off, our approaches perform better than existing uncertain QD methods. This suggests that considering the performance-reproducibility trade-off unlocks important stepping stones that are usually missed when only performance is optimised.

Problem

• Addressing uncertainty in fitness and behavior estimations in QD algorithms.
• Balancing performance and reproducibility as contradictory objectives in QD.
• Developing algorithms to find optimal solutions based on predefined trade-off preferences.

Innovation

• Proposed four new a-priori QD algorithms
• Developed an a-posteriori QD algorithm
• Addressed performance-reproducibility trade-off