An Exploratory Study on AI-driven Visualisation Techniques on Decision Making in Extended Reality

2025-07-15

AI Summary

This study investigates how AI-driven visual interventions—categorized as Inform, Nudge, Recommend, and Instruct—affect user decision-making in XR environments (implemented on Meta Quest Pro using 360° supermarket video). It specifically examines the interaction between intervention intensity and perceived user autonomy. A mixed-methods approach combines quantitative behavioral analysis with semi-structured interviews to assess shifts in preference, decision efficiency, and trust. The study proposes a novel XR-AI collaborative decision-making framework grounded in three principles: autonomy preservation, transparency enhancement, and contextual adaptability. Results indicate that moderate-intensity interventions—particularly Nudge and Recommend—best balance guidance efficacy with user agency. Users significantly prefer systems that transparently disclose AI intent and support counterfactual adjustments. These findings provide empirical evidence and actionable design guidelines for developing trustworthy, human-centered AI visualizations in XR.

Technology Category

• Humans and AI: Explainable AI (XAI) for Human Understanding
• Intelligent Robots: Embodied AI
• Philosophy and Ethics of AI: Safety, Robustness & Trustworthiness

Application Category

• User Modeling, Personalization and Recommendation: Accountability, Transparency, and Ethics for personalization
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

The integration of extended reality (XR) with artificial intelligence (AI) introduces a new paradigm for user interaction, enabling AI to perceive user intent, stimulate the senses, and influence decision-making. We explored the impact of four AI-driven visualisation techniques -- `Inform,' `Nudge,' `Recommend,' and `Instruct' -- on user decision-making in XR using the Meta Quest Pro. To test these techniques, we used a pre-recorded 360-degree video of a supermarket, overlaying each technique through a virtual interface. We aimed to investigate how these different visualisation techniques with different levels of user autonomy impact preferences and decision-making. An exploratory study with semi-structured interviews provided feedback and design recommendations. Our findings emphasise the importance of maintaining user autonomy, enhancing AI transparency to build trust, and considering context in visualisation design.

Problem

• Impact of AI-driven visualization on XR decision-making
• Comparing autonomy levels in visualization techniques
• Balancing user autonomy and AI transparency

Innovation

• AI-driven visualization techniques in XR
• Meta Quest Pro for user decision-making
• Enhancing AI transparency and user autonomy