Vehicle Routing with Finite Time Horizon using Deep Reinforcement Learning with Improved Network Embedding

2026-01-21

AI Summary

This study addresses the finite-time vehicle routing problem (FTVRP) with the objective of maximizing the number of served customers. To this end, the authors propose a Markov decision process formulation that integrates node features, edge features, and adjacency structure, along with a novel context-aware network embedding module that explicitly incorporates remaining time information into both global graph representations and local node embeddings. A policy gradient-based deep reinforcement learning framework is developed to enable efficient solution of the problem. Experimental results demonstrate that the proposed approach significantly improves customer service rates on both real-world and synthetic road networks while achieving faster solution times compared to existing methods.

Technology Category

• Planning, Routing, and Scheduling: Planning with Markov Models (MDPs, POMDPs)
• Search and Optimization: Learning to Search
• Machine Learning: Reinforcement Learning

Application Category

• Graph Algorithms and Modeling for the Web: Graph embeddings and representation learning for Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web

Abstract

In this paper, we study the vehicle routing problem with a finite time horizon. In this routing problem, the objective is to maximize the number of customer requests served within a finite time horizon. We present a novel routing network embedding module which creates local node embedding vectors and a context-aware global graph representation. The proposed Markov decision process for the vehicle routing problem incorporates the node features, the network adjacency matrix and the edge features as components of the state space. We incorporate the remaining finite time horizon into the network embedding module to provide a proper routing context to the embedding module. We integrate our embedding module with a policy gradient-based deep Reinforcement Learning framework to solve the vehicle routing problem with finite time horizon. We trained and validated our proposed routing method on real-world routing networks, as well as synthetically generated Euclidean networks. Our experimental results show that our method achieves a higher customer service rate than the existing routing methods. Additionally, the solution time of our method is significantly lower than that of the existing methods.

Problem

• Vehicle Routing
• Finite Time Horizon
• Customer Service Maximization
• Routing Optimization

Innovation

• network embedding
• finite time horizon
• deep reinforcement learning
• vehicle routing problem
• context-aware graph representation