Quality-Aware Framework for Video-Derived Respiratory Signals

2025-12-16

AI Summary

Video-based respiratory rate (RR) estimation suffers from unreliable performance due to unstable signal quality. To address this, we propose the first segment-level quality-aware predictive fusion framework. It simultaneously extracts ten heterogeneous signals—including facial remote photoplethysmography (rPPG), torso motion, and deep learning features—and jointly applies four spectral estimation algorithms (Welch, MUSIC, FFT, and peak detection) to construct a supervised quality prediction model that dynamically generates segment-level quality indices. These indices drive machine learning–based adaptive signal selection and weighted fusion. The method significantly enhances signal robustness and cross-dataset generalizability. Evaluated on three benchmark datasets—OMuSense-23, COHFACE, and MAHNOB-HCI—it consistently achieves lower RR estimation errors than all individual signal sources and existing fusion approaches.

Technology Category

• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Computer Vision: Video Understanding & Activity Analysis
• Machine Learning: Calibration & Uncertainty Quantification

Application Category

• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Web Mining and Content Analysis: Web data quality in the era of algorithmically-generated content

Abstract

Video-based respiratory rate (RR) estimation is often unreliable due to inconsistent signal quality across extraction methods. We present a predictive, quality-aware framework that integrates heterogeneous signal sources with dynamic assessment of reliability. Ten signals are extracted from facial remote photoplethysmography (rPPG), upper-body motion, and deep learning pipelines, and analyzed using four spectral estimators: Welch's method, Multiple Signal Classification (MUSIC), Fast Fourier Transform (FFT), and peak detection. Segment-level quality indices are then used to train machine learning models that predict accuracy or select the most reliable signal. This enables adaptive signal fusion and quality-based segment filtering. Experiments on three public datasets (OMuSense-23, COHFACE, MAHNOB-HCI) show that the proposed framework achieves lower RR estimation errors than individual methods in most cases, with performance gains depending on dataset characteristics. These findings highlight the potential of quality-driven predictive modeling to deliver scalable and generalizable video-based respiratory monitoring solutions.

Problem

• Unreliable video-based respiratory rate estimation due to inconsistent signal quality
• Integrates heterogeneous signal sources with dynamic reliability assessment
• Enables adaptive signal fusion and quality-based filtering for improved accuracy

Innovation

• Integrates heterogeneous signals with dynamic quality assessment
• Uses machine learning to predict accuracy and select reliable signals
• Enables adaptive signal fusion and quality-based segment filtering