Vibro-Sense: Robust Vibration-based Impulse Response Localization and Trajectory Tracking for Robotic Hands

2026-01-28

AI Summary

This work addresses the challenge of achieving high-precision, whole-body contact perception in robots, which is hindered by the high cost and integration complexity of conventional tactile skins. The authors propose a low-cost alternative by deploying an array of piezoelectric microphones on a robotic hand and leveraging an Audio Spectrogram Transformer to decode contact location and trajectory from vibration signals. They demonstrate for the first time that complex contact dynamics—including material-dependent effects on localization and trajectory—can be effectively inferred from simple vibrational cues alone. The system achieves robust perception even during active robot motion, with static localization errors below 5 mm while maintaining high tracking accuracy under dynamic conditions. The code and dataset are publicly released to support further research.

Technology Category

• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Computer Vision: Motion & Tracking
• Planning, Routing, and Scheduling: Activity and Plan Recognition

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Experiences and lessons learnt from Web-based algorithms and system deployments
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs

Abstract

Rich contact perception is crucial for robotic manipulation, yet traditional tactile skins remain expensive and complex to integrate. This paper presents a scalable alternative: high-accuracy whole-body touch localization via vibro-acoustic sensing. By equipping a robotic hand with seven low-cost piezoelectric microphones and leveraging an Audio Spectrogram Transformer, we decode the vibrational signatures generated during physical interaction. Extensive evaluation across stationary and dynamic tasks reveals a localization error of under 5 mm in static conditions. Furthermore, our analysis highlights the distinct influence of material properties: stiff materials (e.g., metal) excel in impulse response localization due to sharp, high-bandwidth responses, whereas textured materials (e.g., wood) provide superior friction-based features for trajectory tracking. The system demonstrates robustness to the robot's own motion, maintaining effective tracking even during active operation. Our primary contribution is demonstrating that complex physical contact dynamics can be effectively decoded from simple vibrational signals, offering a viable pathway to widespread, affordable contact perception in robotics. To accelerate research, we provide our full datasets, models, and experimental setups as open-source resources.

Problem

• contact perception
• vibration-based localization
• trajectory tracking
• robotic manipulation
• tactile sensing

Innovation

• vibro-acoustic sensing
• impulse response localization
• trajectory tracking
• Audio Spectrogram Transformer
• robotic tactile perception