DHEvo: Data-Algorithm Based Heuristic Evolution for Generalizable MILP Solving

2025-07-21

AI Summary

Existing LLM-driven primal heuristics for mixed-integer linear programming (MILP) suffer from poor generalization and limited transferability across problem instances. To address this, we propose DHEvo—a Data and Heuristic Evolutionary framework—that employs a multi-agent LLM system to jointly optimize a representative set of MILP instances and their corresponding heuristic code. Through iterative cycles of generation, evaluation, and selection of instance-heuristic pairs, DHEvo achieves feature-aware co-evolution. Its key innovation lies in the first integration of instance feature analysis, representative instance selection, and multi-agent LLM-based heuristic generation into an evolutionary computation framework, enabling heuristics to generalize at the problem-class level. On multiple standard MILP benchmarks, DHEvo significantly outperforms both handcrafted heuristics and state-of-the-art LLM-generated methods, delivering substantial improvements in both solution efficiency and robustness.

Technology Category

• Search and Optimization: Evolutionary Computation
• Machine Learning: Evolutionary Learning
• Humans and AI: Human-in-the-loop Machine Learning

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information

Abstract

Primal heuristics play a critical role in improving the efficiency of mixed integer programming (MILP) solvers. As large language models (LLMs) have demonstrated superior code generation abilities, recent MILP works are devoted to leveraging the evolutionary computation approaches with LLMs to generate effective primal heuristics. Although the generated heuristics have achieved better solving performance than the hand-crafted ones with little adaptability, the advantage of current LLM-based methods is limited to few MILP instances in one problem class, as they fail to capture the instance characteristics in the problem class (the MILP instances generated from the same mathematical model are defined as a problem class). Since MILP instances often differ significantly in structure and feature distribution, the neglect of their characteristics in the evolution process results in poor generalization within the same problem class. To overcome this challenge, we propose a data-algorithm co-evolution framework (DHEvo) that iteratively selects representative instances and evolves corresponding heuristics. With the initial instance distribution, we develop an LLM-based multi-agent system to generate data-code pairs simultaneously. These data-code pairs are iteratively refined based on their fitness scores, leading to the identification of the most effective heuristic over the entire problem class. Extensive experiments across diverse MILP benchmarks demonstrate that our approach significantly outperforms both human-designed heuristics and existing LLM-based methods.

Problem

• Improving MILP solver efficiency with generalizable primal heuristics
• Overcoming limited adaptability of LLM-based heuristic generation methods
• Addressing poor generalization across MILP instances within problem classes

Innovation

• Data-algorithm co-evolution framework for MILP
• LLM-based multi-agent system generates data-code pairs
• Iterative refinement using fitness scores for generalization