Aligning LLMs with Graph Neural Solvers for Combinatorial Optimization

2026-03-28

AI Summary

This work addresses the challenge that pure language models struggle to effectively capture the intricate structural relationships inherent in combinatorial optimization problems, which limits their performance on medium- to large-scale instances. To overcome this limitation, the authors propose a unified neuro-heuristic framework that synergistically integrates large language models (LLMs) with graph neural networks (GNNs). For the first time, this approach aligns LLMs and GNNs at both semantic and structural levels, enabling joint encoding of problem textual descriptions and graph-structured data. The resulting model demonstrates significantly enhanced generalization on unseen instances, achieving state-of-the-art performance across multiple combinatorial optimization tasks while exhibiting strong scalability and cross-problem generalization capabilities.

Abstract

Recent research has demonstrated the effectiveness of large language models (LLMs) in solving combinatorial optimization problems (COPs) by representing tasks and instances in natural language. However, purely language-based approaches struggle to accurately capture complex relational structures inherent in many COPs, rendering them less effective at addressing medium-sized or larger instances. To address these limitations, we propose AlignOPT, a novel approach that aligns LLMs with graph neural solvers to learn a more generalizable neural COP heuristic. Specifically, AlignOPT leverages the semantic understanding capabilities of LLMs to encode textual descriptions of COPs and their instances, while concurrently exploiting graph neural solvers to explicitly model the underlying graph structures of COP instances. Our approach facilitates a robust integration and alignment between linguistic semantics and structural representations, enabling more accurate and scalable COP solutions. Experimental results demonstrate that AlignOPT achieves state-of-the-art results across diverse COPs, underscoring its effectiveness in aligning semantic and structural representations. In particular, AlignOPT demonstrates strong generalization, effectively extending to previously unseen COP instances.

Problem

• combinatorial optimization
• large language models
• graph neural networks
• relational structures
• scalability

Innovation

• Large Language Models
• Graph Neural Networks
• Combinatorial Optimization
• Neural Heuristics
• Representation Alignment