EndoWave: Rational-Wavelet 4D Gaussian Splatting for Endoscopic Reconstruction

2025-10-27

AI Summary

In robot-assisted minimally invasive surgery, 3D reconstruction from endoscopic video faces challenges including photometric inconsistency, non-rigid tissue motion, and strong specular reflections. To address these, this paper proposes a spatiotemporally unified 4D Gaussian splatting framework. Methodologically, it introduces optical-flow-guided geometric constraints to enforce temporal consistency and designs a multi-resolution supervision scheme based on rational orthogonal wavelets to jointly optimize appearance and geometry. The approach integrates 4D Gaussian representations, differentiable rendering, optical flow estimation, and wavelet transforms. Evaluated on the real-world EndoNeRF and StereoMIS surgical datasets, our method achieves significant improvements in geometric accuracy and visual fidelity over prior work. Quantitative and qualitative results demonstrate state-of-the-art performance, validating its effectiveness for modeling dynamic endoscopic scenes.

Technology Category

• Computer Vision: 3D Computer Vision
• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation

Abstract

In robot-assisted minimally invasive surgery, accurate 3D reconstruction from endoscopic video is vital for downstream tasks and improved outcomes. However, endoscopic scenarios present unique challenges, including photometric inconsistencies, non-rigid tissue motion, and view-dependent highlights. Most 3DGS-based methods that rely solely on appearance constraints for optimizing 3DGS are often insufficient in this context, as these dynamic visual artifacts can mislead the optimization process and lead to inaccurate reconstructions. To address these limitations, we present EndoWave, a unified spatio-temporal Gaussian Splatting framework by incorporating an optical flow-based geometric constraint and a multi-resolution rational wavelet supervision. First, we adopt a unified spatio-temporal Gaussian representation that directly optimizes primitives in a 4D domain. Second, we propose a geometric constraint derived from optical flow to enhance temporal coherence and effectively constrain the 3D structure of the scene. Third, we propose a multi-resolution rational orthogonal wavelet as a constraint, which can effectively separate the details of the endoscope and enhance the rendering performance. Extensive evaluations on two real surgical datasets, EndoNeRF and StereoMIS, demonstrate that our method EndoWave achieves state-of-the-art reconstruction quality and visual accuracy compared to the baseline method.

Problem

• Addresses photometric inconsistencies and non-rigid motion in endoscopic reconstruction
• Enhances temporal coherence using optical flow-based geometric constraints
• Improves rendering accuracy through multi-resolution rational wavelet supervision

Innovation

• 4D Gaussian Splatting with unified spatio-temporal representation
• Optical flow geometric constraint for temporal coherence
• Multi-resolution rational wavelet supervision for detail enhancement