Gaussian Splatting Feature Fields for Privacy-Preserving Visual Localization

2025-07-31

AI Summary

Addressing the challenge of balancing localization accuracy and privacy preservation in visual localization, this paper proposes Gaussian Lattice Feature Fields (GLFFs), an end-to-end framework integrating explicit geometric modeling with implicit feature learning. Methodologically, it represents scene geometry using 3D Gaussian lattices and constructs a scale-aware 3D feature field via differentiable rendering. Contrastive learning aligns the 3D feature field with outputs from a 2D encoder, while a 3D-structure-guided clustering strategy enables unsupervised mapping from features to semantic segmentation—enabling localization without access to raw images. Experiments demonstrate state-of-the-art accuracy across multiple real-world datasets, significantly improving feature robustness and cross-view generalization. Moreover, by eliminating reliance on original imagery, GLFFs inherently support privacy-preserving localization.

Technology Category

• Computer Vision: Representation Learning for Vision
• Intelligent Robots: Localization, Mapping, and Navigation
• Machine Learning: Large Multimodal Models (LMMs)

Application Category

• Security and Privacy: Data transparency and provenance
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Visual localization is the task of estimating a camera pose in a known environment. In this paper, we utilize 3D Gaussian Splatting (3DGS)-based representations for accurate and privacy-preserving visual localization. We propose Gaussian Splatting Feature Fields (GSFFs), a scene representation for visual localization that combines an explicit geometry model (3DGS) with an implicit feature field. We leverage the dense geometric information and differentiable rasterization algorithm from 3DGS to learn robust feature representations grounded in 3D. In particular, we align a 3D scale-aware feature field and a 2D feature encoder in a common embedding space through a contrastive framework. Using a 3D structure-informed clustering procedure, we further regularize the representation learning and seamlessly convert the features to segmentations, which can be used for privacy-preserving visual localization. Pose refinement, which involves aligning either feature maps or segmentations from a query image with those rendered from the GSFFs scene representation, is used to achieve localization. The resulting privacy- and non-privacy-preserving localization pipelines, evaluated on multiple real-world datasets, show state-of-the-art performances.

Problem

• Develop privacy-preserving visual localization using 3DGS
• Combine explicit geometry with implicit features for accurate localization
• Achieve state-of-the-art performance via pose refinement techniques

Innovation

• 3D Gaussian Splatting for privacy-preserving localization
• Combines explicit geometry with implicit feature fields
• Contrastive framework aligns 3D and 2D feature spaces