SonarSweep: Fusing Sonar and Vision for Robust 3D Reconstruction via Plane Sweeping

2025-11-01

AI Summary

Underwater turbid environments cause severe visual degradation, while sonar suffers from low resolution and inherent blurriness. Existing cross-modal reconstruction methods rely on flawed geometric assumptions, leading to artifacts and poor generalization to complex scenes. To address this, we propose the first end-to-end cross-modal fusion framework that embeds a differentiable plane-sweep algorithm into a deep learning architecture, enabling joint optimization of multi-view images and synchronized sonar data. Our approach eliminates heuristic geometric priors and achieves robust, dense depth estimation via differentiable plane sweeping. Extensive experiments on both synthetic and real-world turbid underwater scenes demonstrate significant improvements in depth accuracy and completeness over state-of-the-art methods. Furthermore, we introduce the first publicly available synchronized binocular vision–sonar dataset, establishing a new benchmark for underwater 3D reconstruction research.

Technology Category

• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Computer Vision: Multi-modal Vision
• Machine Learning: Multimodal Learning

Application Category

• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search
• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs
• Systems and Infrastructure for Web, Mobile and WoT: Experiences and lessons learnt from Web-based algorithms and system deployments

Abstract

Accurate 3D reconstruction in visually-degraded underwater environments remains a formidable challenge. Single-modality approaches are insufficient: vision-based methods fail due to poor visibility and geometric constraints, while sonar is crippled by inherent elevation ambiguity and low resolution. Consequently, prior fusion technique relies on heuristics and flawed geometric assumptions, leading to significant artifacts and an inability to model complex scenes. In this paper, we introduce SonarSweep, a novel, end-to-end deep learning framework that overcomes these limitations by adapting the principled plane sweep algorithm for cross-modal fusion between sonar and visual data. Extensive experiments in both high-fidelity simulation and real-world environments demonstrate that SonarSweep consistently generates dense and accurate depth maps, significantly outperforming state-of-the-art methods across challenging conditions, particularly in high turbidity. To foster further research, we will publicly release our code and a novel dataset featuring synchronized stereo-camera and sonar data, the first of its kind.

Problem

• Achieving accurate 3D reconstruction in degraded underwater environments
• Overcoming limitations of single-modality vision and sonar approaches
• Addressing flaws in prior fusion techniques with heuristics

Innovation

• Fuses sonar and vision via plane sweeping
• Uses end-to-end deep learning framework
• Generates dense accurate depth maps