ExtraGS: Enhancing Endoscopic View Extrapolation via Diffusion-Guided 3D Gaussian Splatting

2026-07-14

AI Summary

This work addresses the challenge of limited endoscopic field-of-view in robot-assisted minimally invasive surgery, which often renders critical anatomical structures invisible. To overcome this, the authors propose a novel view synthesis approach that integrates diffusion models with 3D Gaussian splatting. The method employs uncertainty-guided active sampling via virtual cameras to explore occluded regions and generate plausible pseudo-observations. A confidence-weighted fine-tuning strategy is then applied to refine the reconstruction, preserving reliable areas while significantly enhancing extrapolation quality. Evaluated on multiple public endoscopic datasets, the approach effectively reduces extrapolation artifacts and achieves state-of-the-art performance. Its core innovation lies in the synergistic combination of uncertainty-guided exploration and diffusion prior–driven pseudo-observation generation.

Abstract

Robot-assisted minimally invasive surgery (MIS) critically depends on reliable endoscopic perception for navigation and safety. However, conventional endoscopes provide only a limited field of view, leaving large portions of surrounding anatomy unobserved. Recent neural rendering approaches, such as Neural Radiance Fields and 3D Gaussian Splatting, enable novel view synthesis from endoscopic videos, but their reliance on sparse observations often leads to severe artifacts when extrapolating beyond the training trajectory.In this work, we propose ExtraGS, a framework for enhancing endoscopic view extrapolation via diffusion-guided 3D Gaussian Splatting. Starting from an initial reconstruction, we introduce an uncertainty-guided virtual camera sampling strategy to actively explore blind spots and maximize information gain. The rendered views from these sampled locations are refined using a diffusion model to recover plausible anatomical structures, producing pseudo observations that guide further optimization. To prevent the generated content from degrading reliable regions, we adopt a confidence-weighted fine-tuning strategy when incorporating these pseudo observations.Extensive experiments on multiple public endoscopic datasets demonstrate that ExtraGS significantly reduces extrapolation artifacts and achieves state-of-the-art performance in endoscopic novel view synthesis.

Problem

• endoscopic view extrapolation
• limited field of view
• novel view synthesis
• extrapolation artifacts
• minimally invasive surgery

Innovation

• diffusion-guided
• 3D Gaussian Splatting
• view extrapolation
• uncertainty-guided sampling
• pseudo observations