uScenes: A Multimodal RGB and 3D Sonar Dataset for Underwater Robot Perception

2026-08-27

AI Summary

为解决水下机器人在光线不足和反向散射条件下的感知问题，通过引入包含同步3D声呐点云和RGB图像的多模态数据集uScenes来提高3D场景理解和物体检测精度。

Abstract

Robust perception is essential for the deployment of autonomous underwater robots. However, optical cameras become unreliable under poor illumination and backscatter. Forward looking (2D) acoustic sensors remain effective under these conditions, but they measure range and bearing while leaving elevation unresolved, creating an ambiguity that prevents individual sonar returns from being localized in three dimensional (3D) space. This complicates the sensor use for 3D scene understanding and precise object detection. We introduce \textbf{uScenes}, a multimodal underwater dataset containing synchronized 3D multibeam sonar point clouds and RGB imagery. The dataset contains 110 scenes and 95,834 synchronized observation, representing 277.6 minutes of data collected across multiple field sessions. uScenes establishes a foundation for underwater sensor fusion, cross modal representation learning and 3D scene understanding. Code and datasets are given at https://github.com/era-research-lab/uScenes.

Problem

• underwater robot perception
• poor illumination
• backscatter
• 3D scene understanding
• sensor fusion

Innovation

• Multimodal Dataset
• Underwater Perception
• Sensor Fusion
• Cross-modal Learning