VisReflect: Latent Visual Reflection for Fine-Grained Perception in Long Visual Context

2026-06-29

AI Summary

This work addresses the degradation of fine-grained perception in large vision-language models when processing high-resolution images and long videos, a limitation attributed to visual attention sink effects. To mitigate this, the authors propose a latent visual reflection mechanism that generates query-relevant visual features within a continuous latent space in a single forward pass. This approach dynamically steers attention toward critical regions or frames without requiring explicit localization or additional encoding, thereby circumventing unreliable numerical grounding in discrete token spaces and substantially reducing computational overhead. Experimental results demonstrate that the method outperforms strong baselines across multiple image and video understanding benchmarks, achieving performance gains of 4.1% on image tasks and 1.8% on video tasks, while accelerating inference by approximately 44% compared to conventional cropping-and-resizing strategies.

Abstract

Large Vision Language Models (LVLMs) have achieved remarkable success on vision-language tasks, yet fine-grained perception over high-resolution images and long-context videos remains challenging. As the number of visual tokens increases, the visual attention sink phenomenon becomes increasingly severe, causing irrelevant tokens to absorb a disproportionate amount of attention mass. Recent approaches attempt to mitigate this issue by explicitly predicting bounding boxes or temporal spans and re-encoding the cropped visual regions. Such methods depend on unreliable numeric localization in the discrete token space and incur significant computational overhead due to additional forward passes. In this work, we propose **VisReflect**, a simple yet effective framework that improves fine-grained perception in long visual contexts through latent visual reflection. Instead of decoding intermediate predictions into discrete tokens, the model generates continuous visual reflection that represents question-relevant visual features in the latent space. These reflections selectively emphasize salient regions or frames, guiding attention towards relevant visual tokens within a single forward pass. We conduct comprehensive evaluations on challenging high-resolution image benchmarks, including BLINK, V*, and HRBench-4K/8K, as well as video understanding benchmarks such as MVBench, VideoMME, and MLVU. Our method consistently improves over strong baselines, achieving gains of 4.1% on image benchmarks and 1.8% on video benchmarks. Compared with zooming-based methods, our model achieves comparable performance while reducing inference time by roughly 44% on video understanding.

Problem

• fine-grained perception
• visual attention sink
• long visual context
• high-resolution images
• video understanding

Innovation

• latent visual reflection
• fine-grained perception
• visual attention sink
• long visual context
• vision-language models