Breaking Déjà Vu: Independent Auditing of Visual Place Recognition through Vision-Language Reasoning

2026-07-14

AI Summary

This work addresses the vulnerability of visual place recognition (VPR) in dynamic environments, where reliance on fixed matching thresholds often leads to erroneous loop closures due to the absence of ground-truth labels, thereby compromising the reliability of localization and mapping. To mitigate this issue, the authors propose a general, model-agnostic post-hoc verification framework that, for the first time, integrates vision-language models (VLMs) into VPR auditing. By leveraging cross-modal joint reasoning, the framework performs instance-level match verification between query and candidate images without requiring dataset-specific confidence thresholds, environment priors, or calibrated scores. Evaluated across six benchmark datasets, the method improves recall@1 by an average of 13.6%, reduces the false acceptance rate to 12%, and maintains precision above 95% with coverage exceeding 75%, significantly enhancing the robustness and safety of VPR systems.

Abstract

Visual place recognition (VPR) is a key enabler of accurate localization and long-term autonomous navigation in robotics applications, such as loop closure detection for simultaneous localisation and mapping (SLAM). However, real-world VPR deployment relies on selecting an image matching threshold that balances precision and recall. These thresholds are typically tuned using labeled validation data and fixed during deployment, making them unreliable under environmental changes where ground truth is unavailable. This is particularly problematic in safety-critical robotics, where accepting a false loop closure can corrupt the estimated trajectory and map. In this work, we introduce Visual Place Recognition Auditing, an independent post-retrieval verification framework that leverages Vision-Language Models (VLMs) to assess retrieved matches by reasoning jointly over query and candidate images. Unlike conventional verification methods, our approach performs instance-level verification without requiring architecture-specific confidence measures, dataset-dependent thresholds, or prior knowledge of the deployment environment. We evaluate our method on six benchmark datasets using five state-of-the-art VPR methods and four VLMs. Results show that VLM-based auditing improves recall@1 by 13.6% on average as compared to state-of-the-art methods while reducing false acceptance rates to 12%, maintaining precision above 95% and coverage above 75%.

Problem

• Visual Place Recognition
• False Loop Closure
• Threshold Reliability
• Autonomous Navigation
• Environmental Change

Innovation

• Visual Place Recognition
• Vision-Language Models
• Post-retrieval Verification
• Threshold-free Auditing
• False Loop Closure Detection