Rethinking Pose Refinement in 3D Gaussian Splatting under Pose Prior and Geometric Uncertainty

2026-03-17

AI Summary

This work addresses the sensitivity of 3D Gaussian Splatting (3DGS) pose optimization to initial camera poses and reconstructed geometry, which often leads to reprojection distortions and unstable optimization due to uncertainties in pose priors and geometry. For the first time, this study systematically analyzes these two sources of uncertainty and proposes a relocalization framework that requires neither retraining nor additional supervision. The method explicitly models the pose distribution via Monte Carlo pose sampling and jointly handles uncertainty through a PnP optimization guided by the Fisher Information Matrix. Evaluated on multiple indoor and outdoor benchmarks, the approach significantly improves localization accuracy and demonstrates enhanced optimization stability and robustness under pose and depth noise.

Technology Category

• Intelligent Robots: Localization, Mapping, and Navigation
• Search and Optimization: Sampling/Simulation-based Search
• Machine Learning: Calibration & Uncertainty Quantification

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

3D Gaussian Splatting (3DGS) has recently emerged as a powerful scene representation and is increasingly used for visual localization and pose refinement. However, despite its high-quality differentiable rendering, the robustness of 3DGS-based pose refinement remains highly sensitive to both the initial camera pose and the reconstructed geometry. In this work, we take a closer look at these limitations and identify two major sources of uncertainty: (i) pose prior uncertainty, which often arises from regression or retrieval models that output a single deterministic estimate, and (ii) geometric uncertainty, caused by imperfections in the 3DGS reconstruction that propagate errors into PnP solvers. Such uncertainties can distort reprojection geometry and destabilize optimization, even when the rendered appearance still looks plausible. To address these uncertainties, we introduce a relocalization framework that combines Monte Carlo pose sampling with Fisher Information-based PnP optimization. Our method explicitly accounts for both pose and geometric uncertainty and requires no retraining or additional supervision. Across diverse indoor and outdoor benchmarks, our approach consistently improves localization accuracy and significantly increases stability under pose and depth noise.

Problem

• pose refinement
• 3D Gaussian Splatting
• pose prior uncertainty
• geometric uncertainty
• visual localization

Innovation

• 3D Gaussian Splatting
• pose refinement
• geometric uncertainty
• Monte Carlo sampling
• Fisher Information