Learn to Relax with Large Language Models: Solving Nonlinear Combinatorial Optimization Problems via Bidirectional Coevolution

2025-09-15

AI Summary

Nonlinear combinatorial optimization problems (NCOPs) suffer from multimodality and stringent constraints, rendering conventional methods heavily reliant on expert-designed heuristics and lacking automation. To address these limitations, this paper proposes AutoCO, an end-to-end automated constrained optimization framework. Its key contributions are threefold: (i) the first LLM-driven approach to automatically generate and dynamically evolve constraint relaxation strategies; (ii) a global–local bidirectional co-evolution mechanism—where Monte Carlo tree search enables strategic exploration, LLMs perform structured reasoning and executable code generation, and evolutionary algorithms conduct fine-grained local optimization; and (iii) a unified triple representation scheme that seamlessly bridges strategy generation and execution. Evaluated on three challenging NCOP benchmarks, AutoCO consistently outperforms state-of-the-art baselines, demonstrating superior convergence behavior and solution quality.

Technology Category

• Constraint Satisfaction and Optimization: Constraint Optimization
• Search and Optimization: Combinatorial Optimization
• Reasoning under Uncertainty: Stochastic Optimization

Application Category

• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Nonlinear Combinatorial Optimization Problems (NCOPs) present a formidable computational hurdle in practice, as their nonconvex nature gives rise to multi-modal solution spaces that defy efficient optimization. Traditional constraint relaxation approaches rely heavily on expert-driven, iterative design processes that lack systematic automation and scalable adaptability. While recent Large Language Model (LLM)-based optimization methods show promise for autonomous problem-solving, they predominantly function as passive constraint validators rather than proactive strategy architects, failing to handle the sophisticated constraint interactions inherent to NCOPs.To address these limitations, we introduce the first end-to-end extbf{Auto}mated extbf{C}onstraint extbf{O}ptimization (AutoCO) method, which revolutionizes NCOPs resolution through learning to relax with LLMs.Specifically, we leverage structured LLM reasoning to generate constraint relaxation strategies, which are dynamically evolving with algorithmic principles and executable code through a unified triple-representation scheme. We further establish a novel bidirectional (global-local) coevolution mechanism that synergistically integrates Evolutionary Algorithms for intensive local refinement with Monte Carlo Tree Search for systematic global strategy space exploration, ensuring optimal balance between intensification and diversification in fragmented solution spaces. Finally, comprehensive experiments on three challenging NCOP benchmarks validate AutoCO's consistent effectiveness and superior performance over the baselines.

Problem

• Solving nonlinear combinatorial optimization problems with multi-modal solution spaces
• Automating constraint relaxation strategies using large language models
• Handling sophisticated constraint interactions in nonconvex optimization problems

Innovation

• LLM-generated constraint relaxation strategies
• Triple-representation scheme for dynamic evolution
• Bidirectional coevolution mechanism integrating EA and MCTS