Contextual Multi-Task Reinforcement Learning for Autonomous Reef Monitoring

2026-04-14

AI Summary

Autonomous underwater vehicles (AUVs) face significant control challenges in non-stationary and highly uncertain underwater environments, which hinder their long-term deployment for ecological monitoring tasks such as coral reef surveys. This work proposes a context-aware multi-task reinforcement learning approach that leverages a task-context encoding mechanism to train a single policy capable of simultaneously adapting to multiple related monitoring tasks—such as detecting corals or oysters—within the HoloOcean simulation environment. The proposed method substantially improves sample efficiency, enables strong zero-shot generalization to unseen tasks, and enhances robustness against environmental perturbations like current disturbances, outperforming conventional single-task reinforcement learning baselines.

Technology Category

• Machine Learning: Reinforcement Learning
• Humans and AI: Human-Aware Planning and Behavior Prediction
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

Although autonomous underwater vehicles promise the capability of marine ecosystem monitoring, their deployment is fundamentally limited by the difficulty of controlling vehicles under highly uncertain and non-stationary underwater dynamics. To address these challenges, we employ a data-driven reinforcement learning approach to compensate for unknown dynamics and task variations.Traditional single-task reinforcement learning has a tendency to overfit the training environment, thus, limit the long-term usefulness of the learnt policy. Hence, we propose to use a contextual multi-task reinforcement learning paradigm instead, allowing us to learn controllers that can be reused for various tasks, e.g., detecting oysters in one reef and detecting corals in another. We evaluate whether contextual multi-task reinforcement learning can efficiently learn robust and generalisable control policies for autonomous underwater reef monitoring. We train a single context-dependent policy that is able to solve multiple related monitoring tasks in a simulated reef environment in HoloOcean. In our experiments, we empirically evaluate the contextual policies regarding sample-efficiency, zero-shot generalisation to unseen tasks, and robustness to varying water currents. By utilising multi-task reinforcement learning, we aim to improve the training effectiveness, as well as the reusability of learnt policies to take a step towards more sustainable procedures in autonomous reef monitoring.

Problem

• autonomous underwater vehicles
• reinforcement learning
• multi-task learning
• non-stationary dynamics
• policy generalization

Innovation

• contextual multi-task reinforcement learning
• autonomous underwater vehicles
• zero-shot generalization
• sample efficiency
• robust control