Towards Foundation Models for Mixed Integer Linear Programming

2024-10-10

arXiv.org

AI Summary

Existing MILP-oriented deep learning methods suffer from poor generalization, heavy reliance on expert modeling, and limitations imposed by small, low-diversity datasets. This work introduces the first general-purpose foundation model for Mixed-Integer Linear Programming (MILP). Methodologically, we propose: (1) MILP-Evolve—a novel framework leveraging large language models to evolve high-quality, cross-domain MILP instances; (2) the first formalization of the MILP–natural language semantic alignment task; and (3) a unified architecture integrating graph neural networks, multi-task learning, and instance-level embeddings to enable zero-shot cross-category transfer. Experiments demonstrate consistent superiority over baselines across three core tasks—integrality gap prediction, learning-to-branch, and NL-MILP alignment—with up to 27% improvement on unseen MIPLIB instances. All code and data are publicly released.

Technology Category

• Machine Learning: Deep Neural Architectures and Foundation Models
• Natural Language Processing: Code Generation / Program Synthesis from Natural Language
• Search and Optimization: Evolutionary Computation

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Mixed Integer Linear Programming (MILP) is essential for modeling complex decision-making problems but faces challenges in computational tractability and requires expert formulation. Current deep learning approaches for MILP focus on specific problem classes and do not generalize to unseen classes. To address this shortcoming, we take a foundation model training approach, where we train a single deep learning model on a diverse set of MILP problems to generalize across problem classes. As existing datasets for MILP lack diversity and volume, we introduce MILP-Evolve, a novel LLM-based evolutionary framework that is capable of generating a large set of diverse MILP classes with an unlimited amount of instances. We study our methodology on three key learning tasks that capture diverse aspects of MILP: (1) integrality gap prediction, (2) learning to branch, and (3) a new task of aligning MILP instances with natural language descriptions. Our empirical results show that models trained on the data generated by MILP-Evolve achieve significant improvements on unseen problems, including MIPLIB benchmarks. Our work highlights the potential of moving towards a foundation model approach for MILP that can generalize to a broad range of MILP applications. Our code and data are publicly available at https://github.com/microsoft/OptiGuide.

Problem

• Generalizes across diverse MILP problem classes
• Introduces MILP-Evolve for diverse dataset generation
• Improves performance on unseen MILP benchmarks

Innovation

• Foundation model for MILP
• LLM-based evolutionary framework
• Generalizes across problem classes