A2DEPT: Large Language Model-Driven Automated Algorithm Design via Evolutionary Program Trees

2026-04-27

AI Summary

This work proposes an open-ended algorithm synthesis framework that overcomes the limitations of traditional combinatorial optimization approaches, which heavily rely on expert knowledge, and existing large language model (LLM)-based methods constrained by fixed templates. By casting the LLM as a system-level algorithm architect, the framework automatically synthesizes complete, executable optimization algorithms through tree-structured evolutionary search, a hybrid selection strategy, hierarchical operators, and a lightweight feedback-driven repair mechanism. This approach enables end-to-end algorithmic expression and iterative refinement without template restrictions. Evaluated on both standard and highly constrained benchmarks, the method significantly outperforms current state-of-the-art techniques, reducing the average normalized optimality gap by 9.8% relative to the strongest baseline.

Technology Category

• Search and Optimization: Algorithm Configuration
• Machine Learning: Optimization
• Constraint Satisfaction and Optimization: Search

Application Category

• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information

Abstract

Designing heuristics for combinatorial optimization problems (COPs) is a fundamental yet challenging task that traditionally requires extensive domain expertise. Recently, Large Language Model (LLM)-based Automated Heuristic Design (AHD) has shown promise in autonomously generating heuristic components with minimal human intervention. However, most existing LLM-based AHD methods enforce fixed algorithmic templates to ensure executability, which confines the search to component-level tuning and limits system-level algorithmic expressiveness. To enable open-ended solver synthesis beyond rigid templates, we propose Automated Algorithm Design via Evolutionary Program Trees (A2DEPT), which treats LLMs as system-level algorithm architects. A2DEPT explores the vast program space via a tree-structured evolutionary search with hybrid selection and hierarchical operators, enabling iterative refinement of complete algorithms. To make open-ended generation practical, we enforce executability with a lightweight program-maintenance loop that performs feedback-driven repair. In experiments, A2DEPT consistently outperforms representative LLM-based baselines on both standard and highly constrained benchmarks. On the standard benchmarks, it reduces the mean normalized optimality gap by 9.8% relative to the strongest competing AHD baseline.

Problem

• combinatorial optimization
• automated heuristic design
• algorithm synthesis
• large language models
• evolutionary program trees

Innovation

• Evolutionary Program Trees
• Large Language Models
• Automated Algorithm Design
• Combinatorial Optimization
• Open-ended Solver Synthesis