LLMs for Cold-Start Cutting Plane Separator Configuration

2024-12-16

arXiv.org

AI Summary

Configuring Mixed-Integer Linear Programming (MILP) solver parameters—particularly for cutting-plane separation—is challenging due to high-dimensional, problem-dependent search spaces; existing machine learning approaches suffer from poor generalization, heavy reliance on large-scale labeled data, and difficulty integrating into solver pipelines. Method: We propose the first LLM-driven zero-shot cutting-plane separator configuration framework. It leverages large language models to jointly parse natural-language problem descriptions and LaTeX-based MILP formulations, augmented by literature-informed prompt engineering and semantic modeling of separators—requiring no custom interfaces or extensive retraining. A lightweight, performance-driven clustering ensemble strategy ensures both robustness and real-time responsiveness. Results: On benchmark combinatorial optimization instances and real-world datasets, our method matches state-of-the-art performance while reducing training data requirements by over 90% and generating configurations in under one second.

Technology Category

• Search and Optimization: Mixed Discrete/Continuous Search
• Constraint Satisfaction and Optimization: Solvers and Tools
• Machine Learning: Large Multimodal Models (LMMs)

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

Mixed integer linear programming (MILP) solvers ship with a staggering number of parameters that are challenging to select a priori for all but expert optimization users, but can have an outsized impact on the performance of the MILP solver. Existing machine learning (ML) approaches to configure solvers require training ML models by solving thousands of related MILP instances, generalize poorly to new problem sizes, and often require implementing complex ML pipelines and custom solver interfaces that can be difficult to integrate into existing optimization workflows. In this paper, we introduce a new LLM-based framework to configure which cutting plane separators to use for a given MILP problem with little to no training data based on characteristics of the instance, such as a natural language description of the problem and the associated LaTeX formulation. We augment these LLMs with descriptions of cutting plane separators available in a given solver, grounded by summarizing the existing research literature on separators. While individual solver configurations have a large variance in performance, we present a novel ensembling strategy that clusters and aggregates configurations to create a small portfolio of high-performing configurations. Our LLM-based methodology requires no custom solver interface, can find a high-performing configuration by solving only a small number of MILPs, and can generate the configuration with simple API calls that run in under a second. Numerical results show our approach is competitive with existing configuration approaches on a suite of classic combinatorial optimization problems and real-world datasets with only a fraction of the training data and computation time.

Problem

• Configuring MILP solver parameters is difficult for non-expert users
• Existing ML methods require extensive training data and generalize poorly
• Current approaches are hard to integrate into existing solver workflows

Innovation

• LLM-based framework configures cutting plane separators
• Ensembling strategy clusters candidate configurations into portfolio
• Requires no custom solver interface via simple API calls