Uncertainty Estimation for Novel Views in Gaussian Splatting from Primitive-Based Representations of Error and Visibility

2025-08-04

AI Summary

This work addresses the lack of reliable uncertainty estimation in Gaussian Splatting for novel-view synthesis. We propose a primitive-level error and visibility-aware method for pixel-wise uncertainty estimation. Our core innovation is the first projection of reconstruction errors and visibility information from training views onto individual Gaussian primitives, coupled with explicit foreground-background point cloud separation. Based on this, we design a lightweight regression network that directly predicts pixel-level uncertainty from rendered uncertainty feature maps. The method requires no access to original training data, enabling strong cross-scene generalization. Experiments demonstrate significantly improved correlation between predicted uncertainty and ground-truth rendering error—especially on critical foreground objects—outperforming existing approaches. The proposed framework achieves an effective balance among accuracy, generalizability, and deployment efficiency, making it suitable for trust-critical applications such as robotic navigation and medical imaging.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Reasoning under Uncertainty: Uncertainty Representations
• Intelligent Robots: State Estimation

Application Category

• Security and Privacy: Data transparency and provenance
• Economics, Online Markets and Human Computation: Trust and reliance of crowd workers and data experts on GenAI
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs

Abstract

In this work, we present a novel method for uncertainty estimation (UE) in Gaussian Splatting. UE is crucial for using Gaussian Splatting in critical applications such as robotics and medicine. Previous methods typically estimate the variance of Gaussian primitives and use the rendering process to obtain pixel-wise uncertainties. Our method establishes primitive representations of error and visibility of trainings views, which carries meaningful uncertainty information. This representation is obtained by projection of training error and visibility onto the primitives. Uncertainties of novel views are obtained by rendering the primitive representations of uncertainty for those novel views, yielding uncertainty feature maps. To aggregate these uncertainty feature maps of novel views, we perform a pixel-wise regression on holdout data. In our experiments, we analyze the different components of our method, investigating various combinations of uncertainty feature maps and regression models. Furthermore, we considered the effect of separating splatting into foreground and background. Our UEs show high correlations to true errors, outperforming state-of-the-art methods, especially on foreground objects. The trained regression models show generalization capabilities to new scenes, allowing uncertainty estimation without the need for holdout data.

Problem

• Estimating uncertainty in Gaussian Splatting for novel views
• Improving uncertainty accuracy for critical applications like robotics
• Enhancing foreground object error prediction in splatting techniques

Innovation

• Primitive-based error and visibility representations
• Rendering uncertainty feature maps for novel views
• Pixel-wise regression on holdout data aggregation