Modeling Adaptive Visual Search in Semantically Hierarchical Layouts

2026-06-25

AI Summary

This study investigates how the alignment between semantic grouping and spatial layout enhances visual search efficiency in user interfaces. Building upon a computational rationality framework, it introduces a hierarchical task representation into visual search modeling for the first time, developing a cognitive model that simulates how users integrate semantic structure and visual cues to adapt to task constraints. Through an integrated approach combining computational cognitive modeling, eye-tracking, and semantic categorization experiments, the model successfully replicates human search durations and eye movement patterns. Two user studies further demonstrate that search efficiency significantly improves when semantic groupings align with spatial arrangements. This work provides a computationally grounded evaluation tool for interface design and elucidates the cognitive mechanisms underlying the interplay between semantic and spatial consistency in visual search.

Abstract

This paper introduces a computational cognitive model to investigate how information grouping impacts visual search, a key consideration in user interface design. The model uses computational rationality to view user behavior as an adaptation to cognitive and task constraints. Our work highlights that humans use hierarchical task representations, exploiting semantic and visual structures to improve search efficiency within the constraints of the visual system. We validate this model with data from two human studies focused on visual search and semantic categorization, demonstrating that semantic grouping improves search performance when it aligns with spatial grouping. Our model replicates task durations and eye movement patterns. By improving understanding of how hierarchical memory structures are utilized in human cognition, the model extends previous visual search models. We showcase our model in the rapid prototyping and evaluation of semantic visual groupings within user interface wireframes, suggesting a pathway toward applications in more complex, real-world interface design.

Problem

• visual search
• semantic grouping
• hierarchical layouts
• user interface design
• cognitive modeling

Innovation

• computational rationality
• hierarchical task representation
• semantic grouping
• visual search
• eye movement modeling