WaterSplatting: Fast Underwater 3D Scene Reconstruction Using Gaussian Splatting

2024-08-15

arXiv.org

AI Summary

Underwater 3D scene reconstruction requires joint modeling of scene geometry and the water scattering medium; however, existing methods face fundamental trade-offs: NeRF-based approaches enable coupled geometric and medium modeling but suffer from slow training and non-real-time rendering, while 3D Gaussian Splatting (3DGS) achieves high-speed rendering yet only represents geometry, neglecting volumetric scattering effects. This paper proposes the first dual-representation framework integrating explicit Gaussian geometry with a lightweight voxel-based scattering field. Our method enables decoupled geometric and medium modeling via single-pixel queries, supporting both scattering removal and photorealistic restoration. Leveraging differentiable volumetric scattering modeling and joint optimization, it outperforms state-of-the-art NeRF methods in rendering quality on the SeaThru-NeRF dataset while achieving >30 FPS real-time rendering—marking the first solution for high-fidelity, efficient, and decoupled underwater scene reconstruction.

Technology Category

• Computer Vision: 3D Computer Vision
• Search and Optimization: Mixed Discrete/Continuous Search
• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning

Application Category

• Search and Retrieval-Augmented AI: Vertical and domain-specific search
• Graph Algorithms and Modeling for the Web: Graph embeddings and representation learning for Web-related graphs
• Systems and Infrastructure for Web, Mobile and WoT: Web applications in cross-disciplinary domains and verticals such as mixed reality, smart cities, and digital health

Abstract

The underwater 3D scene reconstruction is a challenging, yet interesting problem with applications ranging from naval robots to VR experiences. The problem was successfully tackled by fully volumetric NeRF-based methods which can model both the geometry and the medium (water). Unfortunately, these methods are slow to train and do not offer real-time rendering. More recently, 3D Gaussian Splatting (3DGS) method offered a fast alternative to NeRFs. However, because it is an explicit method that renders only the geometry, it cannot render the medium and is therefore unsuited for underwater reconstruction. Therefore, we propose a novel approach that fuses volumetric rendering with 3DGS to handle underwater data effectively. Our method employs 3DGS for explicit geometry representation and a separate volumetric field (queried once per pixel) for capturing the scattering medium. This dual representation further allows the restoration of the scenes by removing the scattering medium. Our method outperforms state-of-the-art NeRF-based methods in rendering quality on the underwater SeaThru-NeRF dataset. Furthermore, it does so while offering real-time rendering performance, addressing the efficiency limitations of existing methods. Web: https://water-splatting.github.io

Problem

• Fast underwater 3D scene reconstruction with real-time rendering
• Combining 3D Gaussian Splatting and volumetric rendering for underwater data
• Removing scattering medium effects to restore underwater scenes

Innovation

• Fuses volumetric rendering with 3DGS
• Uses 3DGS for explicit geometry representation
• Separate volumetric field captures scattering medium