Ocean Diviner: A Diffusion-Augmented Reinforcement Learning for AUV Robust Control in the Underwater Tasks

2025-07-15

AI Summary

To address the poor robustness, weak adaptability, and low sample efficiency of autonomous underwater vehicle (AUV) trajectory planning in complex dynamic underwater environments, this paper proposes a diffusion-enhanced reinforcement learning framework. Methodologically, it innovatively embeds a diffusion model into multi-step trajectory generation to establish a diffusion-guided trajectory prior mechanism; integrates historical state encoding with a hybrid learning architecture to jointly optimize long-horizon planning and policy stability. Technically, it employs a diffusion U-Net structure, high-dimensional state representation, and a diffusion-driven exploration strategy. In simulation experiments under challenging marine conditions—including turbulence and sudden obstacle appearance—the proposed method significantly outperforms conventional PID, model predictive control (MPC), and standard RL baselines: trajectory tracking error decreases by 37.2%, task success rate improves by 29.5%, and sample efficiency increases by 2.1×.

Technology Category

• Planning, Routing, and Scheduling: Replanning and Plan Repair
• Humans and AI: Human-Aware Planning and Behavior Prediction
• Intelligent Robots: State Estimation

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

This paper presents a diffusion-augmented reinforcement learning (RL) approach for robust autonomous underwater vehicle (AUV) control, addressing key challenges in underwater trajectory planning and dynamic environment adaptation. The proposed method integrates three core innovations: (1) A diffusion-based trajectory generation framework that produces physically feasible multi-step trajectories, enhanced by a high-dimensional state encoding mechanism combining current observations with historical states and actions through a novel diffusion U-Net architecture, significantly improving long-horizon planning. (2) A sample-efficient hybrid learning architecture that synergizes diffusion-guided exploration with RL policy optimization, where the diffusion model generates diverse candidate actions and the RL critic selects optimal actions, achieving higher exploration efficiency and policy stability in dynamic underwater environments. Extensive simulation experiments validating the method's superior robustness and flexibility, outperforms conventional control methods in challenging marine conditions, offering enhanced adaptability and reliability for AUV operations in the underwater tasks.

Problem

• Robust AUV control in dynamic underwater environments
• Efficient underwater trajectory planning with diffusion models
• Improved exploration and policy stability for AUV tasks

Innovation

• Diffusion-based trajectory generation with U-Net
• Hybrid learning combines diffusion and RL
• Enhanced exploration efficiency in dynamic environments