What to Distinguish and How? Opportunities and Challenges of Augmenting Multiple, Cluttered Objects in Complex Scenes for People with Low Vision

2026-07-12

AI Summary

This work addresses the challenge low-vision users face in extracting critical information from complex visual scenes, a limitation inadequately tackled by existing augmented reality (AR) systems that typically operate in simplified environments or support only single tasks. The paper introduces SceneGlance, a wearable AR system designed for real-world, visually cluttered settings, which pioneers a dynamic visual enhancement mechanism that prioritizes multiple objects based on their contextual importance. Integrating object recognition, importance modeling, and AR rendering, SceneGlance enables hierarchical attention guidance. User studies demonstrate that the system effectively supports low-vision individuals in constructing mental snapshots and adopting hierarchical scanning strategies. However, the findings also reveal a trade-off between focused attention guidance and holistic scene memory, offering critical insights for designing AR assistance in authentic, complex environments.

Abstract

People with low vision (PLV) struggle to perceive complex scenes like busy kitchens and crowded streets, which contain many objects, visual clutter, and dynamic elements. Prior AR systems for low vision either enhance low-level visual features or augment task-relevant objects for single tasks in simple settings, leaving multi-object augmentation in complex scenes underexplored. Informed by a formative study characterizing important objects and their perceived importance for PLV, we built SceneGlance, a wearable AR system that recognizes important objects and visually distinguishes them by importance level. Through a controlled lab study with 12 PLV in a mock-up kitchen scene and a free-form think-aloud study with 13 PLV navigating an outdoor route, we found that AR distinction on object importance shifted PLV's attention toward objects of higher importance, and supported perception strategies such as building mental snapshots from the augmentation distribution and hierarchical scanning by importance. However, this attention shift came with a tradeoff, as augmenting many objects reduced overall scene recall. The studies also surfaced challenges posed by AR augmentations in complex scenes, such as adjacent augmentations blending or interfering with each other, yielding design implications for more practical AR vision enhancement systems in the complex real world.

Problem

• low vision
• complex scenes
• object augmentation
• visual clutter
• importance distinction

Innovation

• augmented reality
• low vision
• object importance
• visual clutter
• wearable system