LOCI: A Locator-Critic with Refinement Loop

2026-08-31

AI Summary

为解决视觉-语言模型在复杂视觉理解任务中的定位问题，提出LOCI框架，通过定位器和评估器的迭代优化提高证据准确性，显著提升模型性能。

Abstract

Vision-Language Models (VLMs) still struggle on tasks requiring complex visual understanding. We argue that the core issue is not high-level reasoning, but instead failing to locate critical details in the image. Due to this shortcoming, VLMs generate often plausible but incorrect reasoning based on flawed perceptual grounding. To address this, we propose Locator-Critic (LOCI), a training-free framework that decouples visual search from evidence verification. LOCI employs a Locator agent to propose candidate visual evidence and a separate Critic agent to evaluate its relevance and sufficiency. These agents engage in an iterative refinement loop, progressively improving the evidence until it is adequate to answer the given question. This decoupled, self-correcting process yields substantial performance gains, achieving state-of-the-art results on multiple complex visual benchmarks. LOCI improves accuracy for both open-weight models like Qwen3-VL (+12.1 on V*, +5.8 on HR-Bench and +11.2 on VisualProbe-Hard) and proprietary models like Gemini 2.5 Pro (+8.9 on V*, +4.3 on HR-Bench, +4.8 on VisualProbe-Hard).

Problem

• Vision-Language Models
• complex visual understanding
• critical details
• perceptual grounding
• incorrect reasoning

Innovation

• Locator-Critic
• Refinement Loop
• Visual Search Decoupling
• Evidence Verification