CORAL-AUV: CFD Oriented Reinforcement Learning for Autonomous Underwater Vehicles

2026-07-10

AI Summary

This study addresses the poor sim-to-real transfer performance of conventional autonomous underwater vehicle (AUV) control methods, which rely on oversimplified hydrodynamic models and struggle to adapt to configuration changes and environmental disturbances. To overcome this limitation, the authors propose a novel framework that integrates computational fluid dynamics (CFD) with reinforcement learning. Specifically, high-fidelity yet computationally efficient surrogate drag models (SDMs) are constructed from CFD data and embedded within a six-degree-of-freedom simulation environment to train control policies. Remarkably, the resulting policy is deployed on a physical AUV without any fine-tuning—demonstrating, for the first time, zero-shot sim-to-real transfer. Compared to controllers based on simplified models, the proposed approach reduces energy consumption by 31%, increases waypoint-to-waypoint speed by 11%, decreases trajectory error by 19%, and is the only method to successfully generalize under parameter perturbations, substantially enhancing both robustness and task performance.

Abstract

Fine grain control and positioning of autonomous underwater vehicles (AUVs) is critical for sampling, maintenance, and survey applications. Traditional control methods for AUVs are labor intensive and are not robust to changes in the vehicle configuration or environmental conditions. Reinforcement learning (RL) promises rapid controller development while handling a range of deployment parameters via domain randomization (DR). However, DR is still limited by the capacity of the underlying simulation to model real physics. In particular, drag physics are difficult to model and are a large contributor to sim-to-real gaps. Meanwhile, computational fluid dynamics (CFD) provides high fidelity drag models but is challenging to leverage within reinforcement learning frameworks due to its computational overhead. Thus, in this paper we exploit the idea of training surrogate approximations of CFD models of a given vehicle, enabling fast inference within RL pipelines. We are the first to successfully deploy a zero-shot RL policy on a 6-DOF AUV in which policy training is performed on surrogate drag models (SDMs) trained on CFD data. We find 31% lower energy usage compared to a controller using simplified physics while traversing between waypoints 11% faster with 19% less error. Our SDM based RL controller better predicts zero-shot transfer and is more robust across reward shaping design choices. When using DR to complete a task with perturbed parameters, we find that the CFD policy is the only controller that successfully transfers. The policies are evaluated in a controlled tank environment and in the field providing extensive testing of the policies' capabilities.

Problem

• Autonomous Underwater Vehicles
• Reinforcement Learning
• Computational Fluid Dynamics
• Sim-to-Real Transfer
• Drag Modeling

Innovation

• Surrogate Drag Models
• Computational Fluid Dynamics
• Reinforcement Learning
• Domain Randomization
• Autonomous Underwater Vehicles