Unsupervised Training of Diffusion Models for Feasible Solution Generation in Neural Combinatorial Optimization

2024-10-15

AI Summary

Existing approaches to bi-set combinatorial optimization problems—such as the Traveling Salesman Problem (TSP), Parallel Machine Scheduling Problem (PMSP), and Asymmetric TSP (ATSP)—rely heavily on hand-crafted heuristics or post-hoc local search, lacking end-to-end autonomous optimization capability. This paper proposes the IC/DC framework: the first unsupervised diffusion modeling paradigm explicitly designed for combinatorial constraints. It introduces a bi-set interactive attention mechanism to explicitly capture complex inter-set dependencies and integrates hard combinatorial constraint enforcement to guarantee solution feasibility. Crucially, IC/DC eliminates reliance on human annotations, heuristic search, or post-processing, enabling fully self-supervised, end-to-end optimization. Empirically, IC/DC achieves state-of-the-art performance on PMSP and ATSP benchmarks, significantly outperforming prior methods that require manual intervention or external guidance.

Technology Category

• Search and Optimization: Combinatorial Optimization
• Constraint Satisfaction and Optimization: Distributed CSP/Optimization
• Reasoning under Uncertainty: Stochastic Optimization

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets

Abstract

Recent advancements in neural combinatorial optimization (NCO) methods have shown promising results in generating near-optimal solutions without the need for expert-crafted heuristics. However, high performance of these approaches often rely on problem-specific human-expertise-based search after generating candidate solutions, limiting their applicability to commonly solved CO problems such as Traveling Salesman Problem (TSP). In this paper, we present IC/DC, an unsupervised CO framework that directly trains a diffusion model from scratch. We train our model in a self-supervised way to minimize the cost of the solution while adhering to the problem-specific constraints. IC/DC is specialized in addressing CO problems involving two distinct sets of items, and it does not need problem-specific search processes to generate valid solutions. IC/DC employs a novel architecture capable of capturing the intricate relationships between items, and thereby enabling effective optimization in challenging CO scenarios. IC/DC achieves state-of-the-art performance relative to existing NCO methods on the Parallel Machine Scheduling Problem (PMSP) and Asymmetric Traveling Salesman Problem (ATSP).

Problem

• Autonomous Learning
• Complex Problem Solving
• Optimization Algorithms

Innovation

• IC/DC
• Diffusion Model
• Complex Optimization