Investigating Seamless Transitions Between Immersive Computational Notebooks and Embodied Data Interactions

2025-09-16

AI Summary

Current immersive analytics tools treat immersive computational notebooks and embodied data exploration systems as disjointed paradigms, forcing analysts to frequently switch between heterogeneous environments—severely disrupting workflow continuity and intuitive interaction. To address this, we propose ICoN, the first prototype system that unifies these two paradigms within a single fully immersive environment. Grounded in embodied cognition theory, ICoN introduces natural multimodal interactions—including gesture-based spatial manipulation and real-time synchronized code execution—that transcend traditional WIMP (Windows, Icons, Menus, Pointer) interface constraints. A controlled user study demonstrates that this unified architecture significantly reduces mode-switching time (by 42% on average) and improves both task completion efficiency and subjective usability (p < 0.01). This work establishes the first operational paradigm for seamless integration in immersive data analysis, advancing the convergence of embodied programming and interactive analytical workflows.

Technology Category

• Cognitive Modeling & Cognitive Systems: Social Cognition And Interaction
• Intelligent Robots: Embodied AI
• Data Mining & Knowledge Management: Conversational Systems for Recommendation & Retrieval

Application Category

• User Modeling, Personalization and Recommendation: Practical large-scale studies of user experience
• Web Mining and Content Analysis: Mining multimedia, multimodal, multilingual, cross-lingual Web data
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets

Abstract

A growing interest in Immersive Analytics (IA) has led to the extension of computational notebooks (e.g., Jupyter Notebook) into an immersive environment to enhance analytical workflows. However, existing solutions rely on the WIMP (windows, icons, menus, pointer) metaphor, which remains impractical for complex data exploration. Although embodied interaction offers a more intuitive alternative, immersive computational notebooks and embodied data exploration systems are implemented as standalone tools. This separation requires analysts to invest considerable effort to transition from one environment to an entirely different one during analytical workflows. To address this, we introduce ICoN, a prototype that facilitates a seamless transition between computational notebooks and embodied data explorations within a unified, fully immersive environment. Our findings reveal that unification improves transition efficiency and intuitiveness during analytical workflows, highlighting its potential for seamless data analysis.

Problem

• Seamless transition between notebooks and embodied interactions
• Unified immersive environment for data analysis workflows
• Improving efficiency and intuitiveness in analytical transitions

Innovation

• Unified immersive environment for notebooks
• Seamless transition between computational and embodied
• Prototype ICoN enhances analytical workflow efficiency