Deep Q-Learning Assisted Bandwidth Reservation for Multi-Operator Time-Sensitive Vehicular Networking

2025-10-21

AI Summary

In multi-operator vehicular networks, existing static or single-objective approaches for bandwidth reservation in safety-critical applications fail to jointly optimize cost-efficiency and resource reliability, while also lacking adaptability to path dynamics and demand–supply uncertainty. Method: This paper proposes a dynamic bandwidth reservation update mechanism based on Double Deep Q-Networks (DDQN), the first to apply DDQN to collaborative bandwidth reservation across multiple operators. It integrates dual objectives—cost minimization and service reliability—via a multi-objective reward function and a carefully designed state representation model, enabling end-to-end policy learning. Contribution/Results: Experiments demonstrate that the proposed method reduces bandwidth cost by 40% compared to greedy strategies and other deep reinforcement learning baselines, while significantly enhancing resource assurance stability and robustness under highly dynamic network conditions.

Technology Category

• Machine Learning: Online Learning & Bandits
• Planning, Routing, and Scheduling: Scheduling under Uncertainty
• Multiagent Systems: Mechanism Design

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Systems and Infrastructure for Web, Mobile and WoT: Virtualization and resource management in Web systems and infrastructures
• Economics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystems

Abstract

Very few available individual bandwidth reservation schemes provide efficient and cost-effective bandwidth reservation that is required for safety-critical and time-sensitive vehicular networked applications. These schemes allow vehicles to make reservation requests for the required resources. Accordingly, a Mobile Network Operator (MNO) can allocate and guarantee bandwidth resources based on these requests. However, due to uncertainty in future reservation time and bandwidth costs, the design of an optimized reservation strategy is challenging. In this article, we propose a novel multi-objective bandwidth reservation update approach with an optimal strategy based on Double Deep Q-Network (DDQN). The key design objectives are to minimize the reservation cost with multiple MNOs and to ensure reliable resource provisioning in uncertain situations by solving scenarios such as underbooked and overbooked reservations along the driving path. The enhancements and advantages of our proposed strategy have been demonstrated through extensive experimental results when compared to other methods like greedy update or other deep reinforcement learning approaches. Our strategy demonstrates a 40% reduction in bandwidth costs across all investigated scenarios and simultaneously resolves uncertain situations in a cost-effective manner.

Problem

• Optimizing bandwidth reservation for time-sensitive vehicular networks
• Minimizing reservation costs across multiple mobile network operators
• Ensuring reliable resource provisioning in uncertain traffic scenarios

Innovation

• Uses Double Deep Q-Network for bandwidth reservation
• Minimizes reservation costs across multiple network operators
• Resolves underbooked and overbooked scenarios cost-effectively