Aquatic-GS: A Hybrid 3D Representation for Underwater Scenes

2024-10-31

arXiv.org

AI Summary

To address the challenge of jointly modeling objects and water media in underwater 3D scenes, this paper proposes a water-object co-optimized hybrid representation: an implicit Neural Water Field (NWF) encodes spatially varying water parameters, while explicit 3D Gaussian Splatting (3DGS) represents object geometry and appearance; both are jointly optimized via a physics-driven underwater imaging model. Key innovations include a pseudo-depth-guided geometric optimization mechanism for enhanced reconstruction accuracy and support for water-removal-based photorealistic appearance recovery. Evaluated on synthetic and real-world datasets, our method achieves superior rendering quality over state-of-the-art approaches and accelerates inference by 410×. For underwater image restoration, it significantly outperforms existing de-watering methods in color correction, fine-detail preservation, and robustness.

Technology Category

• Computer Vision: 3D Computer Vision
• Search and Optimization: Mixed Discrete/Continuous Search
• Constraint Satisfaction and Optimization: Mixed Discrete/Continuous Optimization

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Graph embeddings and representation learning for Web-related graphs

Abstract

Representing underwater 3D scenes is a valuable yet complex task, as attenuation and scattering effects during underwater imaging significantly couple the information of the objects and the water. This coupling presents a significant challenge for existing methods in effectively representing both the objects and the water medium simultaneously. To address this challenge, we propose Aquatic-GS, a hybrid 3D representation approach for underwater scenes that effectively represents both the objects and the water medium. Specifically, we construct a Neural Water Field (NWF) to implicitly model the water parameters, while extending the latest 3D Gaussian Splatting (3DGS) to model the objects explicitly. Both components are integrated through a physics-based underwater image formation model to represent complex underwater scenes. Moreover, to construct more precise scene geometry and details, we design a Depth-Guided Optimization (DGO) mechanism that uses a pseudo-depth map as auxiliary guidance. After optimization, Aquatic-GS enables the rendering of novel underwater viewpoints and supports restoring the true appearance of underwater scenes, as if the water medium were absent. Extensive experiments on both simulated and real-world datasets demonstrate that Aquatic-GS surpasses state-of-the-art underwater 3D representation methods, achieving better rendering quality and real-time rendering performance with a 410x increase in speed. Furthermore, regarding underwater image restoration, Aquatic-GS outperforms representative dewatering methods in color correction, detail recovery, and stability. Our models, code, and datasets can be accessed at https://aquaticgs.github.io.

Problem

• Modeling underwater 3D scenes with object-water coupling
• Integrating implicit water and explicit object representations
• Enhancing underwater scene rendering and restoration accuracy

Innovation

• Hybrid 3D representation with Neural Water Field
• Depth-Guided Optimization for precise geometry
• Physics-based underwater image formation model