Cross, Dwell, or Pinch: Designing and Evaluating Around-Device Selection Methods for Unmodified Smartwatches

2025-03-04

AI Summary

Smartwatches suffer from severely constrained input expressivity due to their small display size. This paper introduces the first hardware-modification-free, ultrasound-based环绕式 (surrounding) finger-tracking and selection interaction system for consumer-grade smartwatches. It enables real-time, fine-grained, one-dimensional finger position tracking in the annular space surrounding the watch face—previously unexplored for input. We systematically evaluate three selection modalities—dual-crossing, hover, and fingertip tap—demonstrating that dual-crossing achieves optimal efficiency for binary tasks, while hover excels in multi-target selection. Tactile feedback significantly improves subjective comfort but yields no statistically significant gains in task performance. Critically, the entire system operates on off-the-shelf commercial hardware without firmware or physical modifications. This work establishes a low-intrusion, high-expressivity interaction paradigm for wearables, extending interactive space beyond the screen boundary.

Technology Category

• Humans and AI: Interaction Techniques and Devices
• Intelligent Robots: Manipulation
• Computer Vision: Motion & Tracking

Application Category

• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Systems and Infrastructure for Web, Mobile and WoT: Web applications in cross-disciplinary domains and verticals such as mixed reality, smart cities, and digital health
• Security and Privacy: Large-scale security measurements

Abstract

Smartwatches offer powerful features, but their small touchscreens limit the expressiveness of the input that can be achieved. To address this issue, we present, and open-source, the first sonar-based around-device input on an unmodified consumer smartwatch. We achieve this using a fine-grained, one-dimensional sonar-based finger-tracking system. In addition, we use this system to investigate the fundamental issue of how to trigger selections during around-device smartwatch input through two studies. The first examines the methods of double-crossing, dwell, and finger tap in a binary task, while the second considers a subset of these designs in a multi-target task and in the presence and absence of haptic feedback. Results showed double-crossing was optimal for binary tasks, while dwell excelled in multi-target scenarios, and haptic feedback enhanced comfort but not performance. These findings offer design insights for future around-device smartwatch interfaces that can be directly deployed on today's consumer hardware.

Problem

• Limited input expressiveness on small smartwatch touchscreens.
• Development of sonar-based around-device input for smartwatches.
• Evaluation of selection methods for around-device smartwatch interaction.

Innovation

• Sonar-based around-device input system
• Fine-grained one-dimensional finger tracking
• Double-crossing and dwell selection methods