OASIS: Real-Time Opti-Acoustic Sensing for Intervention Systems in Unstructured Environments

2025-08-16

AI Summary

To address the challenge of real-time, high-accuracy 3D reconstruction in unstructured underwater environments, this paper proposes OASIS—the first online heterogeneous sensing framework integrating optical imagery and imaging sonar. Leveraging a “hand-in-eye” robotic arm platform, OASIS employs short-baseline multi-view acquisition coupled with an enhanced voxel carving algorithm for real-time multi-view geometric reasoning, overcoming the latency limitations of conventional offline reconstruction. Compared to existing optical-sonar fusion approaches, OASIS achieves millisecond-level updates and centimeter-scale reconstruction accuracy, validated experimentally in a water tank under both autonomous and remotely operated underwater vehicle (ROV) operation scenarios. Key contributions include: (1) the first real-time optical-sonar fusion perception system tailored for unstructured underwater environments; (2) a lightweight, hardware-efficient online 3D reconstruction architecture; and (3) cross-modal data co-modeling and real-time spatial awareness capability.

Technology Category

• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Computer Vision: Remote Sensing / Geospatial AI
• Planning, Routing, and Scheduling: Optimization of Spatio-temporal Systems

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Experiences and lessons learnt from Web-based algorithms and system deployments
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs

Abstract

High resolution underwater 3D scene reconstruction is crucial for various applications, including construction, infrastructure maintenance, monitoring, exploration, and scientific investigation. Prior work has leveraged the complementary sensing modalities of imaging sonars and optical cameras for opti-acoustic 3D scene reconstruction, demonstrating improved results over methods which rely solely on either sensor. However, while most existing approaches focus on offline reconstruction, real-time spatial awareness is essential for both autonomous and piloted underwater vehicle operations. This paper presents OASIS, an opti-acoustic fusion method that integrates data from optical images with voxel carving techniques to achieve real-time 3D reconstruction unstructured underwater workspaces. Our approach utilizes an "eye-in-hand" configuration, which leverages the dexterity of robotic manipulator arms to capture multiple workspace views across a short baseline. We validate OASIS through tank-based experiments and present qualitative and quantitative results that highlight its utility for underwater manipulation tasks.

Problem

• Real-time 3D reconstruction in unstructured underwater environments
• Fusing optical images with sonar for spatial awareness
• Enabling robotic manipulation through opti-acoustic sensing integration

Innovation

• Real-time opti-acoustic fusion method
• Integrates optical images with voxel carving
• Eye-in-hand robotic manipulator configuration