Hierarchical Reinforcement Learning Empowered Task Offloading in V2I Networks

2024-05-18

arXiv.org

AI Summary

This work addresses the dynamic task offloading problem in vehicle-to-infrastructure (V2I) cooperative networks, where tasks exhibit interdependent subtasks structured as directed acyclic graphs (DAGs). We propose a hierarchical offloading framework that jointly optimizes processing latency, on-board energy consumption, and edge computing costs. Our approach innovatively integrates graph attention networks (GATs) to model both intra-DAG and inter-DAG dependencies, and designs a parameterized hierarchical deep reinforcement learning (H-DRL) algorithm capable of handling hybrid action spaces—discrete (offloading decisions) and continuous (resource allocation)—within a unified framework. Extensive experiments on real-world vehicle speed datasets demonstrate that the proposed method significantly reduces overall system cost and consistently outperforms state-of-the-art baselines in jointly optimizing latency and energy efficiency. These results validate the effectiveness and robustness of our framework under dynamic V2I environments.

Technology Category

• Planning, Routing, and Scheduling: Optimization of Spatio-temporal Systems
• Humans and AI: Human-Aware Planning and Behavior Prediction
• Constraint Satisfaction and Optimization: Distributed CSP/Optimization

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Economics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystems

Abstract

Edge computing plays an essential role in the vehicle-to-infrastructure (V2I) networks, where vehicles offload their intensive computation tasks to the road-side units for saving energy and reduce the latency. This paper designs the optimal task offloading policy to address the concerns involving processing delay, energy consumption and edge computing cost. Each computation task consisting of some interdependent sub-tasks is characterized as a directed acyclic graph (DAG). In such dynamic networks, a novel hierarchical Offloading scheme is proposed by leveraging deep reinforcement learning (DRL). The inter-dependencies among the DAGs of the computation tasks are extracted using a graph neural network with attention mechanism. A parameterized DRL algorithm is developed to deal with the hierarchical action space containing both discrete and continuous actions. Simulation results with a real-world car speed dataset demonstrate that the proposed scheme can effectively reduce the system overhead.

Problem

• Design optimal task offloading policy for V2I networks
• Address processing delay, energy consumption, and edge computing cost
• Handle hierarchical action space with discrete and continuous actions

Innovation

• Hierarchical deep reinforcement learning optimizes offloading
• Graph neural network extracts DAG task dependencies
• Parameterized algorithm handles hybrid discrete-continuous actions