Stonefish: Supporting Machine Learning Research in Marine Robotics

2025-02-17

AI Summary

To address the scarcity of real-world underwater data, challenges in manual annotation, and insufficient fidelity of existing simulators for underwater machine learning (ML), this paper introduces the first high-fidelity, open-source simulation platform tailored for marine ML. Methodologically, it pioneers the integration of novel sensors—including event cameras, thermal imagers, and optical flow cameras—models visible-light communication and tether dynamics, and employs physics-based fluid dynamics simulation enhanced with high-accuracy sonar modeling and comprehensive multi-modal sensor simulation. An automated ground-truth annotation framework is also developed. The primary contributions are: (1) establishing the first standardized underwater ML simulation benchmark; (2) significantly improving the sim-to-real transfer performance of perception and navigation algorithms; and (3) enabling efficient development and validation of end-to-end deep learning models for ROVs and AUVs.

Technology Category

• Machine Learning: Hardware-aware ML
• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Search and Optimization: Sampling/Simulation-based Search

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Web Mining and Content Analysis: Web data generation and simulation
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

Simulations are highly valuable in marine robotics, offering a cost-effective and controlled environment for testing in the challenging conditions of underwater and surface operations. Given the high costs and logistical difficulties of real-world trials, simulators capable of capturing the operational conditions of subsea environments have become key in developing and refining algorithms for remotely-operated and autonomous underwater vehicles. This paper highlights recent enhancements to the Stonefish simulator, an advanced open-source platform supporting development and testing of marine robotics solutions. Key updates include a suite of additional sensors, such as an event-based camera, a thermal camera, and an optical flow camera, as well as, visual light communication, support for tethered operations, improved thruster modelling, more flexible hydrodynamics, and enhanced sonar accuracy. These developments and an automated annotation tool significantly bolster Stonefish's role in marine robotics research, especially in the field of machine learning, where training data with a known ground truth is hard or impossible to collect.

Problem

• Enhances marine robotics simulation tools
• Supports machine learning in underwater environments
• Improves sensor and operational modeling accuracy

Innovation

• Added advanced sensor suite
• Improved hydrodynamics modeling
• Enhanced automated annotation tool