Augmented Journeys: Interactive Points of Interest for In-Car Augmented Reality

2025-02-12

AI Summary

With growing passenger engagement in non-driving-related tasks (NDRTs) in vehicular augmented reality (AR), the effectiveness of in-vehicle AR interaction remains unclear. Method: We developed a video-see-through AR system supporting coordinated eye-gaze and pinch gestures to dynamically visualize and multimodally browse geographically anchored points of interest (POIs) outside the vehicle—including both passed and upcoming POIs. Contribution/Results: We conducted the first systematic on-road evaluation of three POI visualization paradigms—list, timeline, and mini-map—via a field study with 21 participants, confirming the list view as optimal for usability. Quantitative analysis revealed significant degradation in 3D spatial interaction accuracy induced by vehicle motion, providing empirical guidance for AR hardware design. Furthermore, we propose evidence-based interaction design principles for NDRT-oriented vehicular AR, advancing natural and robust human–AR collaboration in mobile contexts.

Technology Category

• Planning, Routing, and Scheduling: Optimization of Spatio-temporal Systems
• Computer Vision: Motion & Tracking
• Humans and AI: Interaction Techniques and Devices

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• User Modeling, Personalization and Recommendation: User privacy protection in personalized systems
• Systems and Infrastructure for Web, Mobile and WoT: Web performance, measurement, and characterization

Abstract

As passengers spend more time in vehicles, the demand for non-driving related tasks (NDRTs) increases. In-car Augmented Reality (AR) has the potential to enhance passenger experiences by enabling interaction with the environment through NDRTs using world-fixed Points of Interest (POIs). However, the effectiveness of existing interaction techniques and visualization methods for in-car AR remains unclear. Based on a survey (N=110) and a pre-study (N=10), we developed an interactive in-car AR system using a video see-through head-mounted display to engage with POIs via eye-gaze and pinch. Users could explore passed and upcoming POIs using three visualization techniques: List, Timeline, and Minimap. We evaluated the system's feasibility in a field study (N=21). Our findings indicate general acceptance of the system, with the List visualization being the preferred method for exploring POIs. Additionally, the study highlights limitations of current AR hardware, particularly the impact of vehicle movement on 3D interaction.

Problem

• Enhancing in-car passenger experiences
• Evaluating AR interaction techniques
• Assessing AR hardware limitations

Innovation

• Interactive in-car AR system developed
• Eye-gaze and pinch interaction techniques
• List, Timeline, Minimap visualization methods