POp-GS: Next Best View in 3D-Gaussian Splatting with P-Optimality

2025-03-10

AI Summary

This work addresses the lack of parameter uncertainty modeling in 3D Gaussian Splatting (3D-GS) by proposing the first active perception framework grounded in Optimal Experimental Design (OED). Methodologically, it introduces P/T/D-optimality criteria—previously unexplored in 3D-GS—to formulate an interpretable and computationally tractable information gain metric. A block-diagonal Hessian approximation is proposed to efficiently estimate parameter covariance, enabling quantification of uncertainty reduction and image redundancy induced by novel viewpoints. The framework supports online SLAM tasks, including redundant frame pruning and optimal Next-Best-View (NBV) selection. Experiments on ScanNet and Tanks & Temples demonstrate that the proposed T/D-optimality metric significantly improves NBV selection accuracy, achieving a superior trade-off between reconstruction quality and computational efficiency.

Technology Category

• Search and Optimization: Sampling/Simulation-based Search
• Reasoning under Uncertainty: Stochastic Optimization
• Machine Learning: Active Learning

Application Category

• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics

Abstract

In this paper, we present a novel algorithm for quantifying uncertainty and information gained within 3D Gaussian Splatting (3D-GS) through P-Optimality. While 3D-GS has proven to be a useful world model with high-quality rasterizations, it does not natively quantify uncertainty. Quantifying uncertainty in parameters of 3D-GS is necessary to understand the information gained from acquiring new images as in active perception, or identify redundant images which can be removed from memory due to resource constraints in online 3D-GS SLAM. We propose to quantify uncertainty and information gain in 3D-GS by reformulating the problem through the lens of optimal experimental design, which is a classical solution to measuring information gain. By restructuring information quantification of 3D-GS through optimal experimental design, we arrive at multiple solutions, of which T-Optimality and D-Optimality perform the best quantitatively and qualitatively as measured on two popular datasets. Additionally, we propose a block diagonal approximation of the 3D-GS uncertainty, which provides a measure of correlation for computing more accurate information gain, at the expense of a greater computation cost.

Problem

• Quantify uncertainty in 3D Gaussian Splatting parameters
• Measure information gain from new images in active perception
• Identify redundant images for memory efficiency in online SLAM

Innovation

• Quantify uncertainty in 3D-Gaussian Splatting using P-Optimality.
• Reformulate information gain via optimal experimental design.
• Propose block diagonal approximation for accurate information gain.