Saliency-Aware Multi-Route Thinking: Revisiting Vision-Language Reasoning

2026-02-18

AI Summary

This work addresses the limitations of current vision-language models, which typically incorporate visual information only at the initial stage of reasoning, leading to object hallucinations and unstable inference due to error propagation from early mislocalizations and the absence of dynamic revisiting mechanisms. To overcome this, the authors propose a saliency-aware principle (SAP)-based multi-path reasoning framework that dynamically re-attends to critical visual regions during inference—without requiring additional training—and guides text generation through high-level reasoning principles. The approach enables model-agnostic, parallel path exploration, significantly improving reasoning accuracy and stability while reducing both object hallucination rates and response latency, outperforming conventional chain-of-thought (CoT) methods without any extra training or data.

Technology Category

• Computer Vision: Large Vision Models
• Cognitive Modeling & Cognitive Systems: Conceptual Inference and Reasoning
• Machine Learning: Large Multimodal Models (LMMs)

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Vision-language models (VLMs) aim to reason by jointly leveraging visual and textual modalities. While allocating additional inference-time computation has proven effective for large language models (LLMs), achieving similar scaling in VLMs remains challenging. A key obstacle is that visual inputs are typically provided only once at the start of generation, while textual reasoning (e.g., early visual summaries) is generated autoregressively, causing reasoning to become increasingly text-dominated and allowing early visual grounding errors to accumulate. Moreover, vanilla guidance for visual grounding during inference is often coarse and noisy, making it difficult to steer reasoning over long texts. To address these challenges, we propose \emph{Saliency-Aware Principle} (SAP) selection. SAP operates on high-level reasoning principles rather than token-level trajectories, which enable stable control over discrete generation under noisy feedback while allowing later reasoning steps to re-consult visual evidence when renewed grounding is required. In addition, SAP supports multi-route inference, enabling parallel exploration of diverse reasoning behaviors. SAP is model-agnostic and data-free, requiring no additional training. Empirical results show that SAP achieves competitive performance, especially in reducing object hallucination, under comparable token-generation budgets while yielding more stable reasoning and lower response latency than CoT-style long sequential reasoning.

Problem

• vision-language reasoning
• visual grounding
• object hallucination
• inference-time computation
• reasoning stability

Innovation

• Saliency-Aware Principle
• Vision-Language Reasoning
• Multi-Route Inference
• Visual Grounding
• Model-Agnostic