Evaluating an Immersive Analytics Application at an Enterprise Business Intelligence Customer Conference

2025-08-20

AI Summary

This study addresses the dual challenge in enterprise BI settings of balancing novel user experience with practical workflow utility when evaluating Tableau’s visionOS-based immersive analytics application. We propose and empirically validate an evaluation framework for immersive analytics that integrates user experience with business context requirements. A mixed-methods approach—including task performance metrics, eye-tracking, in-depth interviews, and contextual observation—was employed with 22 enterprise users. Results indicate that head-mounted display (HMD)-enabled spatial 3D interaction enhances multidimensional data exploration efficiency, yet requires careful alignment with existing BI workflows. To our knowledge, this is the first framework to systematically unify three dimensions—usability, utility, and contextual adaptability—providing a reusable methodological paradigm and actionable insights for evaluating XR-based analytics tools in enterprise environments.

Technology Category

• Humans and AI: User Experience and Usability
• Search and Optimization: Evaluation and Analysis
• Data Mining & Knowledge Management: Data Visualization & Summarization

Application Category

• User Modeling, Personalization and Recommendation: Practical large-scale studies of user experience
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Web Mining and Content Analysis: Web data visualization

Abstract

We reflect on an evaluation of an immersive analytics application (Tableau for visionOS) conducted at a large enterprise business intelligence (BI) conference. Conducting a study in such a context offered an opportunistic setting to gather diverse feedback. However, this setting also highlighted the challenge of evaluating usability while also assessing potential utility, as feedback straddled between the novelty of the experience and the practicality of the application in participants' analytical workflows. This formative evaluation with 22 participants allowed us to gather insights with respect to the usability of Tableau for visionOS, along with broader perspectives on the potential for head-mounted displays (HMDs) to promote new ways to engage with BI data. Our experience suggests a need for new evaluation considerations that integrate qualitative and quantitative measures and account for unique interaction patterns with 3D representations and interfaces accessible via an HMD. Overall, we contribute an enterprise perspective on evaluation methodologies for immersive analytics.

Problem

• Evaluating usability and potential utility of immersive BI applications
• Assessing HMDs' role in transforming business intelligence workflows
• Developing evaluation methods for 3D interfaces in enterprise analytics

Innovation

• Evaluated immersive analytics application at enterprise conference
• Used mixed qualitative and quantitative evaluation methods
• Assessed HMD interaction with 3D data representations