RoME: Domain-Robust Mixture-of-Experts for MILP Solution Prediction across Domains

2025-11-04

AI Summary

Existing learning-based methods for accelerating mixed-integer linear programming (MILP) solvers suffer from poor generalization and limited cross-domain transferability. To address this, we propose a task-embedding-driven initial solution prediction framework designed for cross-domain generalization. Our method introduces a Mixture-of-Experts architecture with dynamic instance routing and incorporates two-level distributionally robust optimization: inter-domain alignment and intra-domain perturbation-robust training—enhancing model adaptability to unseen problem distributions. To our knowledge, this is the first approach enabling efficient zero-shot transfer of a single model across diverse domains—including combinatorial optimization, scheduling, and network design. Evaluated on five benchmark domains, it achieves an average speedup of 67.7% over standard solvers. Moreover, on challenging real-world instances from MIPLIB, it significantly outperforms state-of-the-art methods, demonstrating both strong generalization capability and practical utility.

Technology Category

• Search and Optimization: Mixed Discrete/Continuous Search
• Constraint Satisfaction and Optimization: Mixed Discrete/Continuous Optimization
• Machine Learning: Mixture of Experts (MoE)

Application Category

• Graph Algorithms and Modeling for the Web: Graph embeddings and representation learning for Web-related graphs
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Search and Retrieval-Augmented AI: Vertical and domain-specific search

Abstract

Mixed-Integer Linear Programming (MILP) is a fundamental and powerful framework for modeling complex optimization problems across diverse domains. Recently, learning-based methods have shown great promise in accelerating MILP solvers by predicting high-quality solutions. However, most existing approaches are developed and evaluated in single-domain settings, limiting their ability to generalize to unseen problem distributions. This limitation poses a major obstacle to building scalable and general-purpose learning-based solvers. To address this challenge, we introduce RoME, a domain-Robust Mixture-of-Experts framework for predicting MILP solutions across domains. RoME dynamically routes problem instances to specialized experts based on learned task embeddings. The model is trained using a two-level distributionally robust optimization strategy: inter-domain to mitigate global shifts across domains, and intra-domain to enhance local robustness by introducing perturbations on task embeddings. We reveal that cross-domain training not only enhances the model's generalization capability to unseen domains but also improves performance within each individual domain by encouraging the model to capture more general intrinsic combinatorial patterns. Specifically, a single RoME model trained on three domains achieves an average improvement of 67.7% then evaluated on five diverse domains. We further test the pretrained model on MIPLIB in a zero-shot setting, demonstrating its ability to deliver measurable performance gains on challenging real-world instances where existing learning-based approaches often struggle to generalize.

Problem

• Addressing poor generalization of learning-based MILP solvers across domains
• Developing domain-robust framework using mixture-of-experts architecture
• Enhancing cross-domain performance through distributionally robust optimization training

Innovation

• Domain-Robust Mixture-of-Experts framework for MILP
• Dynamic routing to experts using learned task embeddings
• Two-level distributionally robust optimization strategy