Towards High-Resolution Visual Perception via Hierarchical Entity Exploration

2026-07-01

AI Summary

This work addresses the challenge of fine-grained detail loss in high-resolution image perception and the limited generalization of existing methods in complex scenes. To this end, the authors propose a training-free, model-agnostic hierarchical entity exploration framework that reformulates static image understanding as a dynamic, query-guided exploration process. The approach integrates a dual-scoring mechanism, multi-level semantic hierarchy construction, and a confidence-guided backtracking strategy, enabling region evaluation, object detection, entity clustering, and hierarchical semantic organization. It is compatible with diverse multimodal large language models and demonstrates significant performance gains over methods such as ZoomEye and RAP on the Visual Probe and HR-Bench benchmarks, while also exhibiting strong generalization capabilities on MME-RealWorld.

Abstract

High-resolution (HR) image perception remains a key challenge in multimodal large language models (MLLMs), as fine-grained details are often lost when the image is processed as a whole. Existing methods either require training to teach models where to look or heuristically divide the image into fixed regions, both of which struggle to generalize in complex HR scenes. In this work, we propose Hierarchical Entity Exploration (HEE), a training-free and model-agnostic framework that transforms static image understanding into dynamic, query-guided entity exploration. HEE first evaluates each region using a dual scoring mechanism to determine whether it already contains sufficient evidence to answer the question. If not, it applies object detection within the most promising region to extract fine-grained entities, clusters them into coherent subregions, and organizes them into a multi-level semantic hierarchy for deeper exploration. When deeper regions still fail to yield confident answers, a confidence-guided backtracking mechanism revisits alternative paths to ensure adaptive perception. Extensive results show that HEE outperforms training-free methods like ZoomEye and RAP in both accuracy and efficiency on two complex HR benchmarks (Visual Probe and HR-Bench), across different MLLMs such as Qwen2.5-VL and LLaVA-OneVision. Moreover, HEE demonstrates generalization on the MME-RealWorld benchmark.

Problem

• high-resolution image perception
• multimodal large language models
• fine-grained details
• complex HR scenes
• visual understanding

Innovation

• Hierarchical Entity Exploration
• training-free
• query-guided perception
• multi-level semantic hierarchy
• confidence-guided backtracking