QDEvo: A Multi-Objective Quality-Diversity Framework for Automated Heuristic Design

2026-07-06

AI Summary

This work addresses the issue of mode collapse in existing approaches that integrate large language models (LLMs) with evolutionary computation, which often leads to semantically homogeneous heuristics and limited exploratory capacity. To overcome this limitation, the authors propose QDEvo, a novel framework that, for the first time, combines quality-diversity optimization with LLM-driven heuristic generation. QDEvo maintains semantic diversity through an unbounded archive and employs a hierarchical self-reflection mechanism grounded in pretrained code embeddings to guide efficient evolution. Evaluated on both standard benchmarks and real-world industrial scenarios, the method significantly outperforms current state-of-the-art techniques, achieving superior performance in terms of hypervolume and inverted generational distance metrics, while successfully producing diverse ensembles of high-performance, computationally efficient heuristics.

Abstract

The integration of Large Language Models (LLMs) with evolutionary computation has emerged as a powerful paradigm for automated heuristic design in combinatorial optimization. However, existing approaches suffer from mode collapse, converging to homogeneous populations that lack semantic diversity and fail to explore the full algorithmic space. We propose Quality-Diversity Evolution (QDEvo), a multi-objective framework that integrates Quality-Diversity optimization with LLM-driven heuristic search, maintaining an unbounded archive of semantically diverse algorithms using pre-trained code embeddings and incorporating hierarchical self-reflection to guide the evolutionary process. Extensive experiments across standard benchmarks and real-world industrial applications demonstrate that QDEvo significantly outperforms state-of-the-art methods in both Hypervolume and Inverted Generational Distance metrics. Our framework enables the discovery of heuristics that are simultaneously high-performing, computationally efficient, and semantically diverse, providing practitioners with a rich portfolio of solutions for complex optimization problems.

Problem

• mode collapse
• semantic diversity
• automated heuristic design
• combinatorial optimization
• evolutionary computation

Innovation

• Quality-Diversity Optimization
• Large Language Models
• Evolutionary Computation
• Heuristic Design
• Semantic Diversity