EasyUUV: An LLM-Enhanced Universal and Lightweight Sim-to-Real Reinforcement Learning Framework for UUV Attitude Control

2025-10-24

AI Summary

To address poor generalization, weak disturbance rejection, and low deployment efficiency in unmanned underwater vehicle (UUV) attitude control, this paper proposes a lightweight simulation-to-real reinforcement learning (RL) framework integrated with a multimodal large language model (MLLM). Methodologically, we design a parallel RL training pipeline incorporating an adaptive S-Surface hybrid controller; the MLLM online interprets environmental semantics and disturbance characteristics to dynamically adjust control parameters—enabling zero-shot adaptation without retraining. Evaluated on a low-cost 6-DoF UUV platform across diverse real-world underwater scenarios, the framework demonstrates significantly enhanced control robustness and cross-environment generalization. It reduces deployment cost and manual tuning effort while maintaining computational efficiency. This work establishes a novel, resource-efficient, and self-adaptive attitude control paradigm for computationally constrained UUVs.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Intelligent Robots: Behavior Learning & Control
• Planning, Routing, and Scheduling: Planning with Language Models

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information

Abstract

Despite recent advances in Unmanned Underwater Vehicle (UUV) attitude control, existing methods still struggle with generalizability, robustness to real-world disturbances, and efficient deployment. To address the above challenges, this paper presents EasyUUV, a Large Language Model (LLM)-enhanced, universal, and lightweight simulation-to-reality reinforcement learning (RL) framework for robust attitude control of UUVs. EasyUUV combines parallelized RL training with a hybrid control architecture, where a learned policy outputs high-level attitude corrections executed by an adaptive S-Surface controller. A multimodal LLM is further integrated to adaptively tune controller parameters at runtime using visual and textual feedback, enabling training-free adaptation to unmodeled dynamics. Also, we have developed a low-cost 6-DoF UUV platform and applied an RL policy trained through efficient parallelized simulation. Extensive simulation and real-world experiments validate the effectiveness and outstanding performance of EasyUUV in achieving robust and adaptive UUV attitude control across diverse underwater conditions. The source code is available from the following website: https://360zmem.github.io/easyuuv/

Problem

• Improving UUV attitude control generalizability and robustness
• Addressing efficient deployment of reinforcement learning methods
• Enhancing adaptation to unmodeled dynamics and disturbances

Innovation

• LLM-enhanced reinforcement learning framework for UUV control
• Hybrid control combining learned policy with adaptive controller
• Multimodal LLM tunes parameters using visual-textual feedback