Budget-Efficient Automatic Algorithm Design via Code Graph

2026-05-11

AI Summary

Existing approaches to automated algorithm design suffer from poor computational efficiency, often redundantly generating substructures and discarding low-fitness candidates that may harbor valuable components. This work proposes a directed acyclic graph (DAG)-based representation of algorithms, leveraging large language models (LLMs) to produce fine-grained code-editing operations—insertion, deletion, and replacement—and composing historical edits via the graph structure to efficiently explore the algorithmic search space. The method enables credit assignment at the edit level and, supported by theoretical analysis, balances search depth and breadth under limited computational budgets, thereby transcending conventional full-algorithm generation paradigms. Experiments on three combinatorial optimization problems demonstrate significant outperformance over baseline methods under identical token budgets, with enriched context yielding benefits only when the LLM’s prior knowledge is insufficient.

Abstract

Large language models (LLMs) have emerged as powerful tools for automatic algorithm design (AAD). However, existing pipelines remain inefficient. They operate at the granularity of full algorithms, redundantly rewriting recurring substructures and discarding low-fitness candidates that may contain valuable algorithmic features. We formalize budget-efficient automatic algorithm design, wherein the search policy maximizes realized fitness subject to limited computational cost. We propose a directed acyclic graph representation of algorithms and build a search framework that fully exploits the LLM's output. Instead of querying the LLM for full algorithms, we use it to obtain corrections: compact operators that add, replace, or remove code blocks. Each correction augments the graph, yielding new algorithms that compose with prior corrections. This graph structure decomposes algorithms into sets of corrections, enabling correction-level credit assignment that informs subsequent queries. We complement this framework with theoretical insights into the ideal balance between search depth and breadth at different budget levels. We validate our method empirically on three combinatorial optimization problems, demonstrating consistent superiority of our graph-based search over full-algorithm search at equal token budget. Finally, our experiments suggest that rich contexts help only when the LLM's prior knowledge is shallow, and can hinder performance otherwise.

Problem

• automatic algorithm design
• budget efficiency
• combinatorial optimization
• large language models
• algorithm search

Innovation

• code graph
• budget-efficient search
• automatic algorithm design
• correction-based generation
• LLM-guided optimization