How Do We Evaluate Experiences in Immersive Environments?

2026-01-25

AI Summary

Current evaluations of immersive experiences lack a unified standard, suffering from construct overlap, fragmented assessment tools, and insufficient consensus. Addressing this gap, this study conducts a bottom-up scoping review of 375 top-tier conference papers to systematically map the landscape of immersive experience evaluation practices for the first time. The work proposes a user-centered framework grounded in authentic lived experiences, integrating diverse assessment methodologies and advocating for an open, reusable, and comparable evaluation paradigm. It particularly highlights the critical potential of computational modeling in fusing multimodal signals and enabling cross-methodological assessment. By doing so, the study advances a forward-looking research agenda aimed at fostering a sustainable, interdisciplinary ecosystem for evaluating immersive experiences.

Technology Category

• Cognitive Modeling & Cognitive Systems: Computational Creativity
• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Humans and AI: Game Design — Virtual Humans, NPCs and Autonomous Characters

Application Category

• User Modeling, Personalization and Recommendation: Practical large-scale studies of user experience
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Web Mining and Content Analysis: Mining multimedia, multimodal, multilingual, cross-lingual Web data

Abstract

How do we evaluate experiences in immersive environments? Despite decades of research in immersive technologies such as virtual reality, the field remains fragmented. Studies rely on overlapping constructs, heterogeneous instruments, and little agreement on what counts as immersive experience. To better understand this landscape, we conducted a bottom-up scoping review of 375 papers published in ACM CHI, UIST, VRST, SUI, IEEE VR, ISMAR, and TVCG. Our analysis reveals that evaluation practices are often domain- and purpose-specific, shaped more by local choices than by shared standards. Yet this diversity also points to new directions. Instead of multiplying instruments, researchers benefit from integrating and refining them into smarter measures. Rather than focusing only on system outputs, evaluations must center the user's lived experience. Computational modeling offers opportunities to bridge signals across methods, but lasting progress requires open and sustainable evaluation practices that support comparability and reuse. Ultimately, our contribution is to map current practices and outline a forward-looking agenda for immersive experience research.

Problem

• immersive environments
• user experience evaluation
• evaluation standards
• virtual reality
• measurement instruments

Innovation

• immersive experience evaluation
• scoping review
• computational modeling
• user-centered assessment
• evaluation standardization