Large Neighborhood Search meets Iterative Neural Constraint Heuristics

2026-03-21

AI Summary

This work proposes a systematic integration of neural heuristics, such as ConsFormer, into the Large Neighborhood Search (LNS) framework for constraint satisfaction problems. By designing a novel destroy operator guided by model predictions and coupling it with either greedy or sampling-based repair strategies, the approach explicitly steers neighborhood selection. The method significantly outperforms the original neural solver on Sudoku, graph coloring, and MaxCut problems, surpassing both classical and neural baselines. Experimental results further reveal that the combination of random destruction and greedy repair exhibits consistent cross-task superiority, offering a new paradigm for synergistic neural-LNS design.

Technology Category

• Constraint Satisfaction and Optimization: Solvers and Tools
• Search and Optimization: Heuristic Search
• Machine Learning: Neuro-Symbolic Learning

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Neural networks are being increasingly used as heuristics for constraint satisfaction. These neural methods are often recurrent, learning to iteratively refine candidate assignments. In this work, we make explicit the connection between such iterative neural heuristics and Large Neighborhood Search (LNS), and adapt an existing neural constraint satisfaction method-ConsFormer-into an LNS procedure. We decompose the resulting neural LNS into two standard components: the destroy and repair operators. On the destroy side, we instantiate several classical heuristics and introduce novel prediction-guided operators that exploit the model's internal scores to select neighborhoods. On the repair side, we utilize ConsFormer as a neural repair operator and compare the original sampling-based decoder to a greedy decoder that selects the most likely assignments. Through an empirical study on Sudoku, Graph Coloring, and MaxCut, we find that adapting the neural heuristic to an LNS procedure yields substantial gains over its vanilla settings and improves its competitiveness with classical and neural baselines. We further observe consistent design patterns across tasks: stochastic destroy operators outperform greedy ones, while greedy repair is more effective than sampling-based repair for finding a single high-quality feasible assignment. These findings highlight LNS as a useful lens and design framework for structuring and improving iterative neural approaches.

Problem

• Large Neighborhood Search
• Neural Heuristics
• Constraint Satisfaction
• Iterative Refinement
• ConsFormer

Innovation

• Large Neighborhood Search
• Iterative Neural Heuristics
• ConsFormer
• Destroy-Repair Framework
• Neural Constraint Satisfaction