Towards Whole Hand and Wrist Kinematic Tracking with a Wearable A-Mode Ultrasound Probe

2026-06-21

AI Summary

Existing wearable hand motion tracking approaches struggle to simultaneously achieve high degrees of freedom, robustness, and low power consumption, particularly facing bottlenecks in sensor repositioning and on-device deployment. This work addresses these challenges by leveraging the A-mode ultrasound probe-based WULPUS platform and introducing a lightweight multi-output convolutional neural network with only 11,285 parameters, combined with an incremental training strategy. For the first time, it enables end-to-end real-time reconstruction of 23-degree-of-freedom hand-wrist kinematic parameters directly on an nRF52832 microcontroller. The system achieves a per-inference energy cost of 0.73 mJ and latency of 29.1 ms, operates continuously for 36 hours at a total power consumption of 33 mW, reduces wireless bandwidth requirements by 88%, and decreases average absolute error by over 17%, significantly enhancing cross-session generalization and deployment efficiency.

Abstract

A-mode ultrasound (US) has emerged as a promising modality for hand and wrist motion tracking. Prior works have mainly addressed static gesture classification or regression of a few degrees of freedom (DoFs), typically relying on non-wearable systems and external computing devices, and highlight the need for strategies to ensure robustness to sensor repositioning. In this work, we propose a framework for robust whole-hand and wrist kinematic tracking via wearable A-mode US using the WULPUS platform, tackling the regression of 23 DoFs directly on the probe. First, we introduce a compact (11285 parameters) multi-output convolutional neural network combined with an incremental training strategy, which improves inter-session generalization and reduces mean absolute error by more than 17% compared to a non-incremental approach. Second, we demonstrate, for the first time, the feasibility of end-to-end hand and wrist kinematic tracking entirely on-device. We deploy the model on the WULPUS nRF52832 microcontroller, achieving 0.73 mJ per inference, 29.1 ms latency, and showing the feasibility of full operation (data acquisition, online inference, and BLE streaming of results) within 33 mW, enabling up to 36 hours of continuous use and an 88% reduction in wireless bandwidth compared to raw data transmission.

Problem

• hand kinematic tracking
• wrist motion tracking
• wearable ultrasound
• degrees of freedom
• sensor repositioning

Innovation

• wearable ultrasound
• on-device inference
• hand kinematics
• incremental learning
• low-power sensing