PoseCompass: Intelligent Synthetic Pose Selection for Visual Localization

2026-05-12

AI Summary

This work addresses the inefficiency of fine-tuning visual localization models with existing 3D Gaussian splatting–based synthetic data, which often suffers from redundant views and noise. To overcome this limitation, we propose PoseCompass, the first framework to introduce a multidimensional value assessment mechanism that jointly considers localization difficulty, coverage novelty, and rendering observability. This enables intelligent selection of high-information synthetic poses from candidate trajectories, followed by lightweight diffusion alignment for fine-tuning. Our method substantially improves both data augmentation efficiency and pose regression accuracy. On the 7-Scenes dataset, PoseCompass reduces fine-tuning time by a factor of three (from 15.2 to 5.1 minutes) and decreases the median pose error by 53.8%, significantly outperforming random sampling baselines.

Abstract

In visual localization, Absolute Pose Regression (APR) enables real-time 6-DoF camera pose inference from single images, yet critically depends on fine-tuning data quality and coverage. While recent methods leverage 3D Gaussian Splatting (3DGS) for novel view synthesis-based data augmentation, random sampling generates redundant views and noisy samples from poorly reconstructed regions. To mitigate this research gap, we propose PoseCompass, an intelligent pose selection pipeline for 3DGS-based APR. PoseCompass formulates synthetic pose selection and derives a value-based pose ranking mechanism to identify informative poses. The ranking integrates three dimensions: Localization Difficulty, favoring challenging regions; Coverage Novelty, exploring under-sampled areas; and Rendering Observability, filtering artifacts and noise. PoseCompass then generates trajectory-constrained candidates, selects the top-K ranked poses, and synthesizes views using 3DGS with lightweight diffusion-based alignment. Finally, the pose regressor is fine-tuned on mixed real and synthetic data. We evaluate PoseCompass on 7-Scenes, where it reduces adaptation time from 15.2 to 5.1 minutes, a 3x speedup, while cutting median pose errors by 53.8 percent and significantly outperforming random baselines.

Problem

• visual localization
• Absolute Pose Regression
• 3D Gaussian Splatting
• synthetic data augmentation
• pose selection

Innovation

• pose selection
• 3D Gaussian Splatting
• visual localization
• data augmentation
• absolute pose regression