Test-time Scaling over Perception: Resolving the Grounding Paradox in Thinking with Images

2026-04-13

AI Summary

This work addresses the fragility of multimodal large language models in fine-grained visual reasoning, which often stems from the “grounding paradox”—the need to prematurely commit to specific visual regions under insufficient evidence. To overcome this limitation, we propose Test-Time Sensory Scaling (TTSP), a novel framework that reconceptualizes perception as a scalable, test-time inference process. TTSP generates multiple exploratory perceptual trajectories, filters them via entropy-based confidence estimation, and iteratively refines predictions through structured knowledge distillation. By moving beyond conventional static perception paradigms, TTSP achieves substantial performance gains over strong baselines across multiple high-resolution and general multimodal benchmarks, while demonstrating excellent scalability and token efficiency.

Technology Category

• Computer Vision: Multi-modal Vision
• Machine Learning: Multimodal Learning
• Natural Language Processing: Language Grounding & Multi-modal NLP

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Security and Privacy: Large-scale security measurements

Abstract

Recent multimodal large language models (MLLMs) have begun to support Thinking with Images by invoking visual tools such as zooming and cropping during inference. Yet these systems remain brittle in fine-grained visual reasoning because they must decide where to look before they have access to the evidence needed to make that decision correctly. We identify this circular dependency as the Grounding Paradox. To address it, we propose Test-Time Scaling over Perception (TTSP), a framework that treats perception itself as a scalable inference process. TTSP generates multiple exploratory perception traces, filters unreliable traces using entropy-based confidence estimation, distills validated observations into structured knowledge, and iteratively refines subsequent exploration toward unresolved uncertainty. Extensive experiments on high-resolution and general multimodal reasoning benchmarks show that TTSP consistently outperforms strong baselines across backbone sizes, while also exhibiting favorable scalability and token efficiency. Our results suggest that scaling perception at test time is a promising direction for robust multimodal reasoning under perceptual uncertainty.

Problem

• Grounding Paradox
• multimodal reasoning
• visual grounding
• perceptual uncertainty
• fine-grained visual reasoning

Innovation

• Test-Time Scaling
• Perception
• Grounding Paradox
• Multimodal Reasoning
• Entropy-based Confidence