BVS: Bayesian Visual Search with Multimodal Large Language Model for Fine-grained Perception

2026-07-03

AI Summary

Existing multimodal large language models struggle with fine-grained perception in ultra-high-resolution images, particularly exhibiting low efficiency in detecting small objects within cluttered scenes and susceptibility to static prior biases. This work proposes the Bayesian Visual Search (BVS) framework, which formulates visual perception as a global optimization problem over a continuous space-scale manifold. BVS uniquely integrates multimodal reasoning priors with a Gaussian process upper confidence bound (GP-UCB)-driven dynamic posterior correction mechanism. By iteratively refining local observations to mitigate prior noise and recover missing information—and leveraging early-stopping attention unrolling, scale-aware nonstationary kernels, and continuous-space optimization—the method theoretically guarantees a sublinear regret bound. Experiments demonstrate that BVS achieves a significantly superior trade-off between accuracy and efficiency compared to current approaches.

Abstract

While Multimodal Large Language Models (MLLMs) demonstrate impressive general capabilities, they struggle with fine-grained perception in ultra-high-resolution (UHR) images, particularly for tiny objects in cluttered scenes. Existing methods face a dilemma: they either rely on inefficient prior-free scanning, or depend on static prior-driven heuristics that lack posterior correction to rectify initial model biases. To address this, we propose BVS (Bayesian Visual Search), a framework that formulates perception as a global optimization problem over a continuous spatial-scale manifold. Specifically, BVS bridges prior guidance with posterior correction: it utilizes an early-stop attention rollout of MLLM to construct reasoning-aware priors, while employing a scale-aware non-stationary kernel and GP-UCB to dynamically rectify noise and recover missing information in the prior through iterative local observations. We provide theoretical guarantees via sub-linear regret bounds, and extensive experiments demonstrate that BVS significantly outperforms state-of-the-art baselines with a superior trade-off between accuracy and efficiency.

Problem

• fine-grained perception
• ultra-high-resolution images
• tiny objects
• multimodal large language models
• posterior correction

Innovation

• Bayesian Visual Search
• Multimodal Large Language Model
• Fine-grained Perception
• GP-UCB
• Non-stationary Kernel