DISCO: Efficient Diffusion Solver for Large-Scale Combinatorial Optimization Problems

2024-06-28

arXiv.org

AI Summary

To address key bottlenecks in neural solvers for large-scale combinatorial optimization (CO)—namely limited expressive power, low sampling efficiency of diffusion models, and poor generalization—this paper proposes the first efficient diffusion-based framework tailored for CO. Methodologically: (1) we introduce a novel residual-guided constrained sampling mechanism that restricts the denoising process to the feasible solution subspace; (2) we design an analytical, single-step (or minimal-step) reverse process, bypassing iterative denoising; and (3) we incorporate a divide-and-conquer strategy to enable zero-shot generalization across problem scales. Evaluated on Traveling Salesman Problem (TSP) and Maximum Independent Set tasks, our method achieves state-of-the-art performance in solution quality while accelerating inference by up to 5.28× over existing diffusion solvers. It thus simultaneously advances solution quality, computational efficiency, and cross-scale generalizability.

Technology Category

• Search and Optimization: Sampling/Simulation-based Search
• Constraint Satisfaction and Optimization: Distributed CSP/Optimization
• Computer Vision: Diffusion Models for Vision

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Search and Retrieval-Augmented AI: Efficiency and scalability of Web search engines

Abstract

Combinatorial Optimization (CO) problems are fundamentally important in numerous real-world applications across diverse industries, characterized by entailing enormous solution space and demanding time-sensitive response. Despite recent advancements in neural solvers, their limited expressiveness struggles to capture the multi-modal nature of CO landscapes. While some research has shifted towards diffusion models, these models still sample solutions indiscriminately from the entire NP-complete solution space with time-consuming denoising processes, which limit their practicality for large problem scales. We propose DISCO, an efficient DIffusion Solver for large-scale Combinatorial Optimization problems that excels in both solution quality and inference speed. DISCO's efficacy is twofold: First, it enhances solution quality by constraining the sampling space to a more meaningful domain guided by solution residues, while preserving the multi-modal properties of the output distributions. Second, it accelerates the denoising process through an analytically solvable approach, enabling solution sampling with minimal reverse-time steps and significantly reducing inference time. DISCO delivers strong performance on large-scale Traveling Salesman Problems and challenging Maximal Independent Set benchmarks, with inference time up to 5.28 times faster than other diffusion alternatives. By incorporating a divide-and-conquer strategy, DISCO can well generalize to solve unseen-scale problem instances, even surpassing models specifically trained for those scales.

Problem

• Efficiently solving large-scale combinatorial optimization problems
• Improving solution quality by constrained sampling space
• Accelerating denoising process for faster inference speed

Innovation

• Constrains sampling space using solution residues
• Accelerates denoising via analytically solvable approach
• Uses divide-and-conquer for unseen-scale generalization