Deep Reinforcement Learning for Traveling Purchaser Problems

2024-04-03

arXiv.org

AI Summary

The Traveling Procurement Problem (TPP) suffers from strong coupling between routing and procurement decisions, making it challenging for existing methods to balance solution accuracy and computational efficiency. This paper proposes a decoupled deep reinforcement learning (DRL) framework: first, modeling TPP as a bipartite graph and jointly optimizing the global route via DRL and bipartite graph neural networks; second, generating procurement plans efficiently using linear programming. We introduce the first decoupled optimization paradigm for TPP, augmented with a meta-learning strategy enabling generalization across problem scales and demand distributions—even to instances significantly larger than those seen during training. Evaluated on synthetic benchmarks and the TPPLIB dataset, our approach reduces optimality gaps by 40%–90% compared to classical heuristics and achieves substantial speedups on large-scale instances.

Technology Category

• Planning, Routing, and Scheduling: Learning for Planning and Scheduling
• Search and Optimization: Learning to Search
• Constraint Satisfaction and Optimization: Distributed CSP/Optimization

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

The traveling purchaser problem (TPP) is an important combinatorial optimization problem with broad applications. Due to the coupling between routing and purchasing, existing works on TPPs commonly address route construction and purchase planning simultaneously, which, however, leads to exact methods with high computational cost and heuristics with sophisticated design but limited performance. In sharp contrast, we propose a novel approach based on deep reinforcement learning (DRL), which addresses route construction and purchase planning separately, while evaluating and optimizing the solution from a global perspective. The key components of our approach include a bipartite graph representation for TPPs to capture the market-product relations, and a policy network that extracts information from the bipartite graph and uses it to sequentially construct the route. One significant benefit of our framework is that we can efficiently construct the route using the policy network, and once the route is determined, the associated purchasing plan can be easily derived through linear programming, while, leveraging DRL, we can train the policy network to optimize the global solution objective. Furthermore, by introducing a meta-learning strategy, the policy network can be trained stably on large-sized TPP instances, and generalize well across instances of varying sizes and distributions, even to much larger instances that are never seen during training. Experiments on various synthetic TPP instances and the TPPLIB benchmark demonstrate that our DRL-based approach can significantly outperform well-established TPP heuristics, reducing the optimality gap by 40%-90%, and also showing an advantage in runtime, especially on large-sized instances.

Problem

• Solves high computational cost in TPP exact methods
• Improves limited performance of TPP heuristic designs
• Enhances generalization across varying TPP instance sizes

Innovation

• Bipartite graph represents market-product relations
• Policy network constructs route sequentially
• Meta-learning enables generalization across sizes