UAV Trajectory Optimization via Improved Noisy Deep Q-Network

2026-02-05

AI Summary

This work proposes an enhanced Noisy DQN architecture to address the challenges of inefficient exploration and training instability in deep reinforcement learning for unmanned aerial vehicle navigation. The approach integrates residual NoisyLinear layers with an adaptive noise scheduling mechanism to improve exploration capabilities, while incorporating a smooth loss function and soft target network updates to enhance training stability. Evaluated in a 15×15 grid-based navigation environment, the proposed method demonstrates significantly faster convergence compared to standard DQN, achieving a 40% increase in cumulative reward and rapidly attaining the optimal number of steps required to complete the task. This study systematically advances the exploration efficiency and robustness of NoisyNet in trajectory planning applications.

Technology Category

• Planning, Routing, and Scheduling: Replanning and Plan Repair
• Search and Optimization: Learning to Search
• Multiagent Systems: Adversarial Agents

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Economics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystems

Abstract

This paper proposes an Improved Noisy Deep Q-Network (Noisy DQN) to enhance the exploration and stability of Unmanned Aerial Vehicle (UAV) when applying deep reinforcement learning in simulated environments. This method enhances the exploration ability by combining the residual NoisyLinear layer with an adaptive noise scheduling mechanism, while improving training stability through smooth loss and soft target network updates. Experiments show that the proposed model achieves faster convergence and up to $+40$ higher rewards compared to standard DQN and quickly reach to the minimum number of steps required for the task 28 in the 15 * 15 grid navigation environment set up. The results show that our comprehensive improvements to the network structure of NoisyNet, exploration control, and training stability contribute to enhancing the efficiency and reliability of deep Q-learning.

Problem

• UAV trajectory optimization
• deep reinforcement learning
• exploration
• training stability
• Noisy DQN

Innovation

• Noisy Deep Q-Network
• UAV trajectory optimization
• adaptive noise scheduling
• exploration enhancement
• training stability