Robotic Classification of Divers' Swimming States using Visual Pose Keypoints as IMUs

2025-10-15

AI Summary

Traditional human activity recognition methods fail underwater, and real-time monitoring of diver anomalies (e.g., cardiac arrest) remains challenging. To address this, we propose a device-free remote diving behavior recognition method. Leveraging underwater video-driven 3D human pose estimation, we generate high-fidelity joint keypoint motion sequences, which are modeled as pseudo-inertial measurement unit (pseudo-IMU) signals. These signals are fed into a lightweight temporal classification model deployed on an autonomous underwater vehicle (AUV) for real-time swimming state recognition and anomaly detection. Our approach is the first to overcome constraints imposed by underwater wireless communication limitations and the impracticality of physical IMUs, eliminating the need for underwater sensor deployment. In simulated distress scenarios, the system achieves 92.3% accuracy in identifying abnormal swimming patterns, significantly enhancing safety monitoring capabilities for underwater operations.

Technology Category

• Intelligent Robots: State Estimation
• Computer Vision: Video Understanding & Activity Analysis
• Humans and AI: Human-Aware Planning and Behavior Prediction

Application Category

• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

Traditional human activity recognition uses either direct image analysis or data from wearable inertial measurement units (IMUs), but can be ineffective in challenging underwater environments. We introduce a novel hybrid approach that bridges this gap to monitor scuba diver safety. Our method leverages computer vision to generate high-fidelity motion data, effectively creating a ``pseudo-IMU'' from a stream of 3D human joint keypoints. This technique circumvents the critical problem of wireless signal attenuation in water, which plagues conventional diver-worn sensors communicating with an Autonomous Underwater Vehicle (AUV). We apply this system to the vital task of identifying anomalous scuba diver behavior that signals the onset of a medical emergency such as cardiac arrest -- a leading cause of scuba diving fatalities. By integrating our classifier onboard an AUV and conducting experiments with simulated distress scenarios, we demonstrate the utility and effectiveness of our method for advancing robotic monitoring and diver safety.

Problem

• Monitors scuba diver safety using visual pose keypoints as pseudo-IMUs
• Overcomes wireless signal attenuation in underwater environments
• Identifies anomalous diver behavior signaling medical emergencies

Innovation

• Generates pseudo-IMU data from 3D joint keypoints
• Uses computer vision to overcome underwater signal issues
• Integrates classifier on AUV for diver safety monitoring