A High-Throughput Platform to Bench Test Smartphone-Based Heart Rate Measurements Derived From Video

2025-06-29

AI Summary

To address poor cross-device compatibility and inconsistent evaluation in smartphone-based photoplethysmography (PPG) heart rate monitoring—stemming from device fragmentation—this paper introduces the first high-throughput, automated benchmarking platform capable of parallel testing across 12 devices. We propose a novel synthetic PPG video generation method that enables precise control over ground-truth heart rate and signal quality. Integrated with multi-device synchronization, video playback-driven stimulation, and automated data acquisition, the platform employs clinically validated algorithms to ensure standardized bench testing. Experimental evaluation spans 20 mainstream smartphone models; under ANSI/CTA compliance criteria, the platform achieves 100% accuracy in heart rate classification. It attains a mean absolute percentage error of only 0.11% for heart rate estimation and a PPG waveform correlation coefficient of 0.92. This work establishes a reproducible, scalable, and cross-platform performance evaluation paradigm for mobile health applications.

Technology Category

• Computer Vision: Biometrics, Face, Gesture & Pose
• Machine Learning: Hardware-aware ML
• Humans and AI: Interaction Techniques and Devices

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Web performance, measurement, and characterization
• Security and Privacy: Large-scale security measurements
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics

Abstract

Smartphone-based heart rate (HR) monitoring apps using finger-over-camera photoplethysmography (PPG) face significant challenges in performance evaluation and device compatibility due to device variability and fragmentation. Manual testing is impractical, and standardized methods are lacking. This paper presents a novel, high-throughput bench-testing platform to address this critical need. We designed a system comprising a test rig capable of holding 12 smartphones for parallel testing, a method for generating synthetic PPG test videos with controllable HR and signal quality, and a host machine for coordinating video playback and data logging. The system achieved a mean absolute percentage error (MAPE) of 0.11% +/- 0.001% between input and measured HR, and a correlation coefficient of 0.92 +/- 0.008 between input and measured PPG signals using a clinically-validated smartphone-based HR app. Bench-testing results of 20 different smartphone models correctly classified all the devices as meeting the ANSI/CTA accuracy standards for HR monitors (MAPE <10%) when compared to a prospective clinical study with 80 participants, demonstrating high positive predictive value. This platform offers a scalable solution for pre-deployment testing of smartphone HR apps to improve app performance, ensure device compatibility, and advance the field of mobile health.

Problem

• Evaluating smartphone heart rate app performance and compatibility
• Developing high-throughput testing for PPG-based HR measurements
• Ensuring mobile health apps meet ANSI/CTA accuracy standards

Innovation

• High-throughput bench-testing platform for smartphone HR apps
• Synthetic PPG test videos with controllable HR
• Parallel testing of 12 smartphones simultaneously