GauSS-MI: Gaussian Splatting Shannon Mutual Information for Active 3D Reconstruction

2025-04-29

AI Summary

This work addresses the limitation of existing next-best-view (NBV) selection methods in real-time active 3D reconstruction, which prioritize geometric completeness while neglecting visual rendering uncertainty. We propose GauSS-MI—the first pixel-level visual-quality-oriented uncertainty metric—by embedding Shannon mutual information into the 3D Gaussian Splatting framework, enabling differentiable, real-time, and semantics-aware NBV planning. Our method jointly optimizes viewpoint selection and camera motion through probabilistic modeling and active perception. Evaluated across diverse synthetic and real-world scenes, GauSS-MI achieves significant improvements in visual reconstruction quality (↑ PSNR/SSIM), reduces required sampling by 37%, and maintains >20 FPS real-time performance—thereby overcoming the fundamental constraints of conventional geometry-driven paradigms.

Technology Category

• Computer Vision: 3D Computer Vision
• Reasoning under Uncertainty: Uncertainty Representations
• Search and Optimization: Sampling/Simulation-based Search

Application Category

• Security and Privacy: Large-scale security measurements
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

This research tackles the challenge of real-time active view selection and uncertainty quantification on visual quality for active 3D reconstruction. Visual quality is a critical aspect of 3D reconstruction. Recent advancements such as Neural Radiance Fields (NeRF) and 3D Gaussian Splatting (3DGS) have notably enhanced the image rendering quality of reconstruction models. Nonetheless, the efficient and effective acquisition of input images for reconstruction-specifically, the selection of the most informative viewpoint-remains an open challenge, which is crucial for active reconstruction. Existing studies have primarily focused on evaluating geometric completeness and exploring unobserved or unknown regions, without direct evaluation of the visual uncertainty within the reconstruction model. To address this gap, this paper introduces a probabilistic model that quantifies visual uncertainty for each Gaussian. Leveraging Shannon Mutual Information, we formulate a criterion, Gaussian Splatting Shannon Mutual Information (GauSS-MI), for real-time assessment of visual mutual information from novel viewpoints, facilitating the selection of next best view. GauSS-MI is implemented within an active reconstruction system integrated with a view and motion planner. Extensive experiments across various simulated and real-world scenes showcase the superior visual quality and reconstruction efficiency performance of the proposed system.

Problem

• Real-time active view selection for 3D reconstruction
• Quantifying visual uncertainty in reconstruction models
• Selecting informative viewpoints using Shannon Mutual Information

Innovation

• Probabilistic model quantifies visual uncertainty per Gaussian
• GauSS-MI assesses visual mutual information in real-time
• Active reconstruction system integrates view and motion planner