AoI-Aware Task Offloading and Transmission Optimization for Industrial IoT Networks: A Branching Deep Reinforcement Learning Approach

2025-10-18

AI Summary

To address the joint optimization challenge of age-of-information (AoI)-aware task offloading and resource allocation in multi-base-station real-time monitoring for industrial IoT (IIoT), this paper proposes a deep reinforcement learning framework based on Branching Dueling Double DQN. Our method innovatively introduces a branching action structure to reduce the complexity of the high-dimensional combinatorial decision space from exponential to linear, while integrating Hessian matrix-based semidefiniteness analysis for coordinated bandwidth and computational resource allocation. Theoretical analysis guarantees policy convergence and feasibility of resource allocation. Experimental results demonstrate that the proposed approach achieves 75% faster convergence compared to state-of-the-art deep RL and heuristic methods, and reduces long-term average AoI by at least 22%, significantly enhancing system timeliness and status freshness.

Technology Category

• Planning, Routing, and Scheduling: Optimization of Spatio-temporal Systems
• Intelligent Robots: Learning & Optimization for ROB
• Search and Optimization: Learning to Search

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Economics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystems
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation

Abstract

In the Industrial Internet of Things (IIoT), the frequent transmission of large amounts of data over wireless networks should meet the stringent timeliness requirements. Particularly, the freshness of packet status updates has a significant impact on the system performance. In this paper, we propose an age-of-information (AoI)-aware multi-base station (BS) real-time monitoring framework to support extensive IIoT deployments. To meet the freshness requirements of IIoT, we formulate a joint task offloading and resource allocation optimization problem with the goal of minimizing long-term average AoI. Tackling the core challenges of combinatorial explosion in multi-BS decision spaces and the stochastic dynamics of IIoT systems is crucial, as these factors render traditional optimization methods intractable. Firstly, an innovative branching-based Dueling Double Deep Q-Network (Branching-D3QN) algorithm is proposed to effectively implement task offloading, which optimizes the convergence performance by reducing the action space complexity from exponential to linear levels. Then, an efficient optimization solution to resource allocation is proposed by proving the semi-definite property of the Hessian matrix of bandwidth and computation resources. Finally, we propose an iterative optimization algorithm for efficient joint task offloading and resource allocation to achieve optimal average AoI performance. Extensive simulations demonstrate that our proposed Branching-D3QN algorithm outperforms both state-of-the-art DRL methods and classical heuristics, achieving up to a 75% enhanced convergence speed and at least a 22% reduction in the long-term average AoI.

Problem

• Minimizing long-term average Age-of-Information in IIoT networks
• Optimizing joint task offloading and resource allocation decisions
• Solving combinatorial explosion in multi-base station decision spaces

Innovation

• Branching-D3QN algorithm reduces action space complexity
• Proves Hessian matrix property for resource allocation
• Iterative optimization for joint task and resource management