Task-specific Subnetwork Discovery in Reinforcement Learning for Autonomous Underwater Navigation

2026-04-23

AI Summary

This study addresses the challenge of deploying multitask reinforcement learning for underwater autonomous navigation, where limited interpretability hinders practical application. Leveraging the HoloOcean simulation platform, the authors analyze a pretrained policy network and identify task-specific subnetworks activated during distinct navigation tasks. Their findings reveal that only approximately 1.5% of the network’s weights are responsible for task differentiation, with 85% of these concentrated between the contextual input layer and the first hidden layer—highlighting the critical role of contextual variables in task specialization. This approach substantially enhances policy interpretability and offers a promising avenue for efficient transfer and continual learning in complex, dynamic environments.

Technology Category

• Machine Learning: Reinforcement Learning
• Humans and AI: Human-Aware Planning and Behavior Prediction
• Search and Optimization: Learning to Search

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Economics, Online Markets and Human Computation: Social networks and social learning

Abstract

Autonomous underwater vehicles are required to perform multiple tasks adaptively and in an explainable manner under dynamic, uncertain conditions and limited sensing, challenges that classical controllers struggle to address. This demands robust, generalizable, and inherently interpretable control policies for reliable long-term monitoring. Reinforcement learning, particularly multi-task RL, overcomes these limitations by leveraging shared representations to enable efficient adaptation across tasks and environments. However, while such policies show promising results in simulation and controlled experiments, they yet remain opaque and offer limited insight into the agent's internal decision-making, creating gaps in transparency, trust, and safety that hinder real-world deployment. The internal policy structure and task-specific specialization remain poorly understood. To address these gaps, we analyze the internal structure of a pretrained multi-task reinforcement learning network in the HoloOcean simulator for underwater navigation by identifying and comparing task-specific subnetworks responsible for navigating toward different species. We find that in a contextual multi-task reinforcement learning setting with related tasks, the network uses only about 1.5% of its weights to differentiate between tasks. Of these, approximately 85% connect the context-variable nodes in the input layer to the next hidden layer, highlighting the importance of context variables in such settings. Our approach provides insights into shared and specialized network components, useful for efficient model editing, transfer learning, and continual learning for underwater monitoring through a contextual multi-task reinforcement learning method.

Problem

• interpretable reinforcement learning
• multi-task reinforcement learning
• autonomous underwater navigation
• subnetwork discovery
• decision transparency

Innovation

• task-specific subnetworks
• multi-task reinforcement learning
• contextual RL
• interpretable AI
• autonomous underwater navigation