HyFinBall: a Hybrid User Interface for Coordinated 2D+3D Visualization in Semi-Immersive VR

2025-12-10

AI Summary

To address input modality mismatch in 2D/3D co-interactive visualization within semi-immersive VR, this paper proposes HyFinBall—a hybrid interface introducing the novel “Buttonball” dual-mode adaptive cursor mapping mechanism. This enables seamless switching among four interaction modalities under bimanual coordination: 6-DOF free-space navigation, planar-constrained 3-DOF manipulation, multi-touch, and 3D gestures. It further presents the first systematic empirical validation of cross-dimensional interaction efficiency advantages within mixed UIs. We implement a coordinated 2D+3D visualization system on a desktop VR platform. Two user studies demonstrate that HyFinBall achieves an average 27.4% improvement in task efficiency and a 31.6% reduction in error rate compared to baseline methods, significantly enhancing the experience of multidimensional data co-analysis.

Technology Category

• Computer Vision: Multi-modal Vision
• Humans and AI: Interaction Techniques and Devices
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• User Modeling, Personalization and Recommendation: User modeling and simulation for interactive and conversational systems
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

Sophisticated 3D visualization applications usually provide coordinated 2D and 3D views. Normally 3D input device is used for 3D tasks since they perform better than traditional 2D input devices. However, they do not perform better for 2D tasks. This paper presents a bimanual hybrid user interface that supports four interaction modes: a dual 6-degree-of-freedom (DOF) input device mode, a dual planar constrained 3DOF input device mode, a dual 2-finger multi-touch mode, and 3D hand and finger gestures. The application is a multi-dimensional visualization with coordinated 3D and 2D views on a desktop VR system. The input devices are buttonballs with seamless switching between 3D and 2D device modes, as well as between free-hand finger input and device usage. The 3D and 2D device mode switch automatically switches a buttonball's visual representation between a 3D cursor and a 2D cursor while changing the available user interaction techniques between 3D and 2D interaction techniques to interact with the coordinated views. The paper also provides two formal user studies to evaluate HyFinBall for various dimensional tasks, including 3D, 2D, and cross-dimensional tasks. Our experimental results show the benefits of the HyFinBall interface for cross-dimensional tasks that require 3D and 2D interactions.

Problem

• Develops a hybrid UI for coordinated 2D+3D visualization in semi-immersive VR.
• Enables seamless switching between 3D and 2D input modes for diverse tasks.
• Evaluates the interface's effectiveness for cross-dimensional 3D and 2D interactions.

Innovation

• Hybrid bimanual interface with four interaction modes
• Seamless switching between 3D and 2D input device modes
• Automatic cursor and interaction technique adaptation for coordinated views