BlinkBud: Detecting Hazards from Behind via Sampled Monocular 3D Detection on a Single Earbud

2025-12-01

AI Summary

To address the challenge of real-time perception of rearward hazardous vehicles for pedestrians and cyclists, this paper proposes a low-power, ear-worn device–smartphone collaborative 3D sensing system. Methodologically, it integrates monocular 3D object detection, pose-angle-compensated coordinate alignment, Kalman filter–based trajectory estimation, and a reinforcement learning–driven sparse image sampling strategy—effectively mitigating head-motion interference while balancing energy efficiency and accuracy. The system achieves average power consumption of only 29.8 mW on the ear-worn device and 702.6 mW on the smartphone; hazardous vehicle detection attains a false positive rate of 4.90% and a false negative rate of 1.47%. Its core contribution lies in the first integration of reinforcement learning–guided dynamic image sampling with ear-worn multimodal collaborative computing, enabling high-accuracy, low-latency, and sustainable rearward road hazard perception.

Technology Category

• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Computer Vision: Motion & Tracking
• Machine Learning: Hardware-aware ML

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Energy management for devices in mobile Web and WoT environments
• Security and Privacy: Large-scale security measurements
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation

Abstract

Failing to be aware of speeding vehicles approaching from behind poses a huge threat to the road safety of pedestrians and cyclists. In this paper, we propose BlinkBud, which utilizes a single earbud and a paired phone to online detect hazardous objects approaching from behind of a user. The core idea is to accurately track visually identified objects utilizing a small number of sampled camera images taken from the earbud. To minimize the power consumption of the earbud and the phone while guaranteeing the best tracking accuracy, a novel 3D object tracking algorithm is devised, integrating both a Kalman filter based trajectory estimation scheme and an optimal image sampling strategy based on reinforcement learning. Moreover, the impact of constant user head movements on the tracking accuracy is significantly eliminated by leveraging the estimated pitch and yaw angles to correct the object depth estimation and align the camera coordinate system to the user's body coordinate system, respectively. We implement a prototype BlinkBud system and conduct extensive real-world experiments. Results show that BlinkBud is lightweight with ultra-low mean power consumptions of 29.8 mW and 702.6 mW on the earbud and smartphone, respectively, and can accurately detect hazards with a low average false positive ratio (FPR) and false negative ratio (FNR) of 4.90% and 1.47%, respectively.

Problem

• Detects hazardous objects approaching from behind using a single earbud and phone.
• Minimizes power consumption while ensuring accurate 3D object tracking with sampled images.
• Compensates for user head movements to improve tracking accuracy and safety.

Innovation

• Uses single earbud and phone for real-time hazard detection
• Integrates Kalman filter and reinforcement learning for optimal tracking
• Corrects depth with head movement estimation to improve accuracy