Rethinking LLM-Driven Heuristic Design: Generating Efficient and Specialized Solvers via Dynamics-Aware Optimization

2026-01-14

arXiv.org

AI Summary

This work addresses the limitations of existing large language model (LLM)-driven heuristic design, which often relies solely on end-point evaluation while neglecting solution process efficiency and incurs high re-adaptation costs under distribution shifts. To overcome these issues, the authors propose DASH, a novel framework that introduces a convergence-dynamics-aware evaluation mechanism to jointly optimize search strategies and runtime scheduling. DASH further incorporates a profiled library retrieval module that enables efficient reuse across heterogeneous problem instances through solver profiling. Experimental results demonstrate that DASH achieves over a fourfold improvement in runtime efficiency across four combinatorial optimization problems and outperforms state-of-the-art methods in overall performance. Moreover, under distribution shift scenarios, DASH significantly reduces solution quality degradation and lowers LLM re-adaptation costs by approximately 90%.

Technology Category

• Search and Optimization: Heuristic Search
• Planning, Routing, and Scheduling: Planning with Language Models
• Constraint Satisfaction and Optimization: Search

Application Category

• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

Large Language Models (LLMs) have advanced the field of Combinatorial Optimization through automated heuristic generation. Instead of relying on manual design, this LLM-Driven Heuristic Design (LHD) process leverages LLMs to iteratively generate and refine solvers to achieve high performance. However, existing LHD frameworks face two critical limitations: (1) Endpoint-only evaluation, which ranks solvers solely by final gap to a reference solution, ignoring the convergence process and runtime efficiency; (2) High adaptation costs, where distribution shifts necessitate re-adaptation to generate specialized solvers for heterogeneous instance groups. To address these issues, we propose Dynamics-Aware Solver Heuristics (DASH), a framework that co-optimizes solver search mechanisms and runtime schedules guided by a convergence-aware metric, thereby identifying efficient and high-performance solvers. Furthermore, to mitigate expensive re-adaptation, DASH incorporates Profiled Library Retrieval (PLR), which maintains group-specialized solvers for profile-aware warm starts. These solvers are archived concurrently during evolution, allowing DASH to reuse matched specialists across heterogeneous distributions without restarting adaptation. Experiments on four combinatorial optimization problems demonstrate that DASH improves runtime efficiency by over 4 times while outperforming prior LHD baselines in the overall balance between gap and runtime across diverse problem scales. Furthermore, by enabling profile-aware warm starts, DASH maintains lower gap under distribution shift while reducing LLM adaptation costs by about 90%.

Problem

• Combinatorial Optimization
• LLM-Driven Heuristic Design
• Runtime Efficiency
• Distribution Shift
• Solver Adaptation

Innovation

• Dynamics-Aware Optimization
• LLM-Driven Heuristic Design
• Convergence-Aware Metric
• Profiled Library Retrieval
• Solver Specialization