G-LNS: Generative Large Neighborhood Search for LLM-Based Automatic Heuristic Design

2026-02-09

AI Summary

This work proposes G-LNS, a novel framework that leverages large language models (LLMs) to co-evolve destroy-repair operator pairs within large neighborhood search for combinatorial optimization. Unlike existing LLM-based heuristic design methods constrained by fixed structures and prone to local optima, G-LNS introduces a cooperative evaluation mechanism that captures the interaction logic between operators, enabling effective perturbation and reconstruction of solution structures. This approach transcends the structural limitations of traditional heuristics and substantially enhances global exploration. Evaluated on benchmark problems such as the Traveling Salesman Problem (TSP) and Capacitated Vehicle Routing Problem (CVRP), G-LNS outperforms both state-of-the-art LLM-based heuristic methods and classical solvers, achieving near-optimal solutions with lower computational overhead and demonstrating strong generalization on unseen instances.

Technology Category

• Search and Optimization: Heuristic Search
• Planning, Routing, and Scheduling: Planning with Language Models
• Machine Learning: Large Multimodal Models (LMMs)

Application Category

• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

While Large Language Models (LLMs) have recently shown promise in Automated Heuristic Design (AHD), existing approaches typically formulate AHD around constructive priority rules or parameterized local search guidance, thereby restricting the search space to fixed heuristic forms. Such designs offer limited capacity for structural exploration, making it difficult to escape deep local optima in complex Combinatorial Optimization Problems (COPs). In this work, we propose G-LNS, a generative evolutionary framework that extends LLM-based AHD to the automated design of Large Neighborhood Search (LNS) operators. Unlike prior methods that evolve heuristics in isolation, G-LNS leverages LLMs to co-evolve tightly coupled pairs of destroy and repair operators. A cooperative evaluation mechanism explicitly captures their interaction, enabling the discovery of complementary operator logic that jointly performs effective structural disruption and reconstruction. Extensive experiments on challenging COP benchmarks, such as Traveling Salesman Problems (TSP) and Capacitated Vehicle Routing Problems (CVRP), demonstrate that G-LNS significantly outperforms LLM-based AHD methods as well as strong classical solvers. The discovered heuristics not only achieve near-optimal solutions with reduced computational budgets but also exhibit robust generalization across diverse and unseen instance distributions.

Problem

• Automated Heuristic Design
• Combinatorial Optimization Problems
• Large Neighborhood Search
• Local Optima
• Structural Exploration

Innovation

• Generative Large Neighborhood Search
• LLM-based Heuristic Design
• Co-evolution of Operators
• Combinatorial Optimization
• Automated Heuristic Design