DiMo-GUI: Advancing Test-time Scaling in GUI Grounding via Modality-Aware Visual Reasoning
Natural language query-based GUI element localization faces challenges including visual element diversity, spatial clutter, and linguistic ambiguity. This paper proposes a training-free modality-separation and dynamic focusing framework: it decouples textual and icon representations, leverages a general-purpose vision-language model, and integrates candidate region generation with a progressive zooming mechanism to enable independent, modality-specific reasoning. Furthermore, hierarchical visual focusing is introduced, wherein spatial attention dynamically refines the localization region. To our knowledge, this is the first approach achieving modality-aware semantic alignment and fine-grained localization for GUI elements without any task-specific training. Evaluated on standard GUI benchmarks, our method significantly outperforms existing baselines, demonstrating both the effectiveness and generalizability of the co-designed modality separation and region focusing strategy.