Adapting Vision Foundation Models to Acoustics for Pose-Free 3D Sonar Reconstruction

2026-09-05

AI Summary

研究通过利用视觉基础模型与声学之间的几何关系及物理噪声模型生成合成数据，解决了缺乏大规模声纳数据集的问题，实现了基于声纳的无姿态3D重建。

Abstract

Vision foundation models trained on Internet-scale RGB datasets enable remarkable capabilities across a range of tasks, from text-to-video generation to few-shot 3D scene reconstruction. An acoustic foundation model trained on large-scale sonar datasets could enable similar capabilities in the underwater domain, where turbidity and low-visibility conditions make conventional RGB foundation models inapplicable. Unfortunately, a lack of freely available large-scale sonar datasets makes training such a model from scratch impractical. In this work, we demonstrate that vision foundation models can be efficiently adapted to the sonar setting by (1) exploiting the geometric relationship between the two sensing modalities and (2) employing accurate physics-based noise models for synthetic data generation. The resulting sonar adaptation models enable new capabilities: For the first time, we experimentally demonstrate sonar-based pose-free 3D reconstruction.

Problem

• sonar datasets
• underwater domain
• low-visibility conditions
• foundation models

Innovation

• vision foundation models
• sonar adaptation
• pose-free 3D reconstruction
• synthetic data generation