Soft Quality-Diversity Optimization

2025-11-30

AI Summary

Traditional quality-diversity (QD) algorithms rely on explicit discretization of behavior spaces, leading to the curse of dimensionality and severe memory bottlenecks in high-dimensional and large-scale solution spaces. To address this, we propose “Soft QD”, a novel framework that eliminates explicit discretization and introduces, for the first time, a formal definition of QD optimization with guaranteed monotonicity and asymptotic equivalence to the standard QD score. Building upon this foundation, we derive SQUAD—a fully differentiable algorithm that enforces approximate behavioral diversity as a continuous regularizer, enabling end-to-end gradient-based optimization. Empirical evaluation demonstrates that SQUAD matches state-of-the-art performance on canonical QD benchmarks while achieving substantial improvements in scalability and computational efficiency on high-dimensional tasks.

Technology Category

• Search and Optimization: Learning to Search
• Intelligent Robots: Learning & Optimization for ROB
• Humans and AI: Human-Aware Planning and Behavior Prediction

Application Category

• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web

Abstract

Quality-Diversity (QD) algorithms constitute a branch of optimization that is concerned with discovering a diverse and high-quality set of solutions to an optimization problem. Current QD methods commonly maintain diversity by dividing the behavior space into discrete regions, ensuring that solutions are distributed across different parts of the space. The QD problem is then solved by searching for the best solution in each region. This approach to QD optimization poses challenges in large solution spaces, where storing many solutions is impractical, and in high-dimensional behavior spaces, where discretization becomes ineffective due to the curse of dimensionality. We present an alternative framing of the QD problem, called emph{Soft QD}, that sidesteps the need for discretizations. We validate this formulation by demonstrating its desirable properties, such as monotonicity, and by relating its limiting behavior to the widely used QD Score metric. Furthermore, we leverage it to derive a novel differentiable QD algorithm, emph{Soft QD Using Approximated Diversity (SQUAD)}, and demonstrate empirically that it is competitive with current state of the art methods on standard benchmarks while offering better scalability to higher dimensional problems.

Problem

• Optimizes diverse high-quality solutions without discretization
• Addresses scalability in large high-dimensional solution spaces
• Introduces a differentiable algorithm for improved performance

Innovation

• Soft QD sidesteps discretization in behavior spaces
• SQUAD algorithm is differentiable and scalable
• Soft QD relates to QD Score metric properties