GSVisLoc: Generalizable Visual Localization for Gaussian Splatting Scene Representations

2025-08-25

AI Summary

This work addresses camera pose estimation for visual localization in scenes reconstructed using 3D Gaussian Splatting (3DGS), proposing the first general-purpose method that requires no model modification, retraining, or auxiliary reference images. The approach operates in three stages: (i) coarse-grained scene feature construction via downsampled 3D Gaussian encoding; (ii) fine-grained cross-modal feature matching guided by image patch encodings; and (iii) joint optimization for pose refinement. Its core innovation lies in the first end-to-end utilization of explicit 3DGS representations for visual localization—enabling direct, geometry-aware pose estimation without implicit neural rendering or feature distillation. The method demonstrates strong cross-scene generalization. Evaluated on standard indoor and outdoor benchmarks, it achieves significantly higher localization accuracy than existing 3DGS-based approaches, validating both its effectiveness and robustness under diverse real-world conditions.

Technology Category

• Computer Vision: 3D Computer Vision
• Intelligent Robots: State Estimation
• Search and Optimization: Local Search

Application Category

• Graph Algorithms and Modeling for the Web: Graph embeddings and representation learning for Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation

Abstract

We introduce GSVisLoc, a visual localization method designed for 3D Gaussian Splatting (3DGS) scene representations. Given a 3DGS model of a scene and a query image, our goal is to estimate the camera's position and orientation. We accomplish this by robustly matching scene features to image features. Scene features are produced by downsampling and encoding the 3D Gaussians while image features are obtained by encoding image patches. Our algorithm proceeds in three steps, starting with coarse matching, then fine matching, and finally by applying pose refinement for an accurate final estimate. Importantly, our method leverages the explicit 3DGS scene representation for visual localization without requiring modifications, retraining, or additional reference images. We evaluate GSVisLoc on both indoor and outdoor scenes, demonstrating competitive localization performance on standard benchmarks while outperforming existing 3DGS-based baselines. Moreover, our approach generalizes effectively to novel scenes without additional training.

Problem

• Estimating camera position and orientation from query images
• Matching scene features to image features robustly
• Generalizing visual localization to novel scenes without retraining

Innovation

• Generalizable visual localization for Gaussian Splatting
• Robust matching of encoded 3D Gaussian features
• Three-step coarse-to-fine pose estimation refinement