One Forward Beats Two: InnerZoom for Accurate and Efficient GUI Grounding

2026-06-29

AI Summary

Existing MLLM-based GUI grounding methods struggle to effectively propagate intermediate-layer object-aware information to the final coordinate prediction, limiting their accuracy. This work proposes InnerZoom, a framework that dynamically extracts, preserves, and reinjects target-relevant cues through a cross-layer evidence bridging mechanism within a single forward pass to guide autoregressive coordinate generation. InnerZoom achieves grounding accuracy on par with or surpassing the two-stage ZoomIn approach while significantly reducing latency and computational overhead—marking the first single-stage architecture to do so. Evaluated across six GUI benchmarks, InnerZoom-4B attains state-of-the-art performance, yielding an average improvement of 5.3 points (1.3 points over ZoomIn), with 31.8% lower latency and 29% fewer computations.

Abstract

MLLM-based GUI grounding methods commonly formulate target localization as autoregressive coordinate generation, enabling models to leverage the strong instruction-following and semantic understanding capabilities of MLLMs. However, this formulation requires the model to retain region-level target evidence while decoding coordinate tokens with the spatial precision demanded by GUI clicking. Our diagnostic analysis reveals that target-region awareness emerges in intermediate decoder layers but is neither retained nor translated into the final coordinate prediction. Existing ZoomIn-style methods address this issue through an external crop-and-rerun pass, which improves localization but increases end-to-end latency and computational cost. To retain the accuracy benefits of two-pass zooming without this extra cost, we propose InnerZoom, a single-forward framework for cross-layer evidence bridging. InnerZoom transforms target-related cues from the original forward pass into a compact cross-layer evidence state, then preserves, refines, and reinjects this state throughout later decoding layers to guide coordinate prediction. Extensive experimental results suggest that InnerZoom-4B achieves state-of-the-art performance on all six GUI grounding benchmarks, obtaining 64.7 on OSWorld-G, 40.2 on UI-Vision, 73.1 on OSWorld-GR, and 87.6 on MMBench-GUI, surpassing the previous best results by 4.1, 3.2, 2.9, and 2.3 points, respectively. Under a controlled 4B setting, InnerZoom improves the same SFT+RL baseline by 5.3 points on average and outperforms two-pass ZoomIn by 1.3 points on average, while reducing end-to-end latency by up to 31.8% and TFLOPs by about 29%. Code and models will be publicly available.

Problem

• GUI grounding
• coordinate generation
• target localization
• spatial precision
• computational efficiency

Innovation

• InnerZoom
• GUI grounding
• cross-layer evidence bridging
• single-forward inference
• MLLM