Disambiguating 2D-3D Correspondences in Gaussian Splatting-based Feature Fields for Visual Localization

2026-05-08

AI Summary

This work addresses the instability and lack of multi-view consistency in existing photometric optimization-based Gaussian splatting methods for visual localization, which stem from ambiguous 2D–3D correspondences. To resolve this, the authors propose SplitGS-Loc, a novel framework that introduces a Gaussian mixture model–based splitting strategy to decouple each Gaussian into multiple sub-Gaussians, thereby enabling precise one-to-one 2D–3D matching. Additionally, the method leverages rasterization-based combination weights to select features that are both discriminative and consistent across views. Notably, SplitGS-Loc requires neither scene-specific fine-tuning nor iterative optimization, yet achieves substantial improvements in accuracy and robustness on standard visual localization benchmarks, demonstrating its effectiveness and practicality under zero-shot conditions.

Abstract

While Gaussian Splatting-based Feature Fields (GSFFs) have shown promise for visual localization, this paper highlights that photometrically optimized GSFFs are inherently ill-suited for 2D-3D matching. The volumetric extent of each Gaussian induces many-to-one pixel-to-point mappings that destabilize PnP-based pose estimation, while photometric optimization gives rise to superfluous Gaussians devoid of multi-view consistency. To address these issues, we propose SplitGS-Loc, a localization-specialized GSFFs construction framework that disambiguates 2D-3D correspondences by exploiting Gaussian attributes. Our key design, Mixture-of-Gaussians-based splitting, decomposes each Gaussian into smaller Gaussians, replacing ambiguous many-to-one with precise one-to-one correspondences. In parallel, we exploit composition weights from GS rasterization to select Gaussians that significantly and consistently contribute across multiple views and aggregate discriminative features through strong pixel-Gaussian associations, enforcing multi-view consistency. The resulting compact yet discriminative feature fields enable stable PnP convergence, achieving state-of-the-art performance on localization benchmarks. Extensive experiments validate that SplitGS-Loc extends the utility of photometric GSFFs to accurate and efficient localization by exploiting Gaussian attributes, without per-scene training or iterative pose refinement.

Problem

• Gaussian Splatting
• Visual Localization
• 2D-3D Correspondence
• Multi-view Consistency
• Feature Fields

Innovation

• Gaussian Splatting
• 2D-3D correspondence disambiguation
• Mixture-of-Gaussians splitting
• multi-view consistency
• visual localization