ConstraintLLM: A Neuro-Symbolic Framework for Industrial-Level Constraint Programming

2025-10-07

AI Summary

Automated modeling of constraint optimization problems (COPs) remains underdeveloped, and constraint programming (CP) modeling research lags behind operations research (OR). Method: We propose ConstraintLLM—the first large language model (LLM) specialized for CP tasks—integrating a constraint-aware retrieval module (CARM), a tree-of-thoughts (ToT) reasoning framework, and symbolic solvers to enable guided self-correction. It is built via multi-instruction supervised fine-tuning on open-source LLMs and evaluated on IndusCP, the first industrial-scale CP benchmark. Contribution/Results: ConstraintLLM achieves state-of-the-art (SOTA) solution accuracy on mainstream CP benchmarks and doubles modeling accuracy over baselines on IndusCP. It significantly improves reliability and generalization in formal COP modeling for industrial applications.

Technology Category

• Constraint Satisfaction and Optimization: Constraint Programming
• Machine Learning: Large Multimodal Models (LMMs)
• Cognitive Modeling & Cognitive Systems: Symbolic Representations

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

Constraint programming (CP) is a crucial technology for solving real-world constraint optimization problems (COPs), with the advantages of rich modeling semantics and high solving efficiency. Using large language models (LLMs) to generate formal modeling automatically for COPs is becoming a promising approach, which aims to build trustworthy neuro-symbolic AI with the help of symbolic solvers. However, CP has received less attention compared to works based on operations research (OR) models. We introduce ConstraintLLM, the first LLM specifically designed for CP modeling, which is trained on an open-source LLM with multi-instruction supervised fine-tuning. We propose the Constraint-Aware Retrieval Module (CARM) to increase the in-context learning capabilities, which is integrated in a Tree-of-Thoughts (ToT) framework with guided self-correction mechanism. Moreover, we construct and release IndusCP, the first industrial-level benchmark for CP modeling, which contains 140 challenging tasks from various domains. Our experiments demonstrate that ConstraintLLM achieves state-of-the-art solving accuracy across multiple benchmarks and outperforms the baselines by 2x on the new IndusCP benchmark. Code and data are available at: https://github.com/william4s/ConstraintLLM.

Problem

• Automates constraint programming modeling using large language models
• Enhances in-context learning with constraint-aware retrieval mechanisms
• Addresses industrial-level constraint optimization problems across diverse domains

Innovation

• LLM fine-tuned for constraint programming modeling
• Constraint-aware retrieval with guided self-correction
• Industrial benchmark with 140 domain-specific tasks