Contactless Respiratory Monitoring on Heterogeneous Mobile Robots: A Multimodal Edge-Computing Framework

2026-06-15

AI Summary

This work addresses the challenge of robust non-contact respiratory rate monitoring in high-risk scenarios such as emergency response, where existing methods are vulnerable to variations in illumination, subject posture, and robotic platform heterogeneity. The paper proposes the first general-purpose multimodal edge computing framework tailored for heterogeneous mobile robots, which adaptively fuses RGB, thermal, near-infrared (NIR), and low-light imaging modalities based on ambient brightness. By integrating keypoint-guided chest region extraction and signal quality index (SQI)-based filtering, the system achieves reliable monitoring without requiring platform-specific customization or parameter retuning. Validated across three distinct robotic platforms, the approach supports monitoring distances up to 8 meters in RGB and low-light modes, 6 meters in NIR, and short-range thermal sensing—even operating effectively in complete darkness—thereby establishing, for the first time, clear applicability boundaries for each modality under varying environmental conditions.

Abstract

Respiratory-rate (RR) monitoring is a critical component of remote triage and victim assessment in emergency response, disaster recovery, and infectious-disease scenarios, where minimizing physical contact can reduce responder risk and improve operational safety. However, field deployment of contactless RR monitoring remains challenging due to variable illumination, posture changes, platform heterogeneity, and the impracticality of wearable sensors in hazardous environments. In this paper, we present a modality-adaptive contactless RR monitoring framework for heterogeneous mobile robots with onboard edge computing. The proposed system combines brightness-adaptive sensor selection across RGB, thermal, near-infrared (NIR), and low-light cameras, keypoint-guided chest ROI extraction for posture-robust monitoring, and a signal-quality-index (SQI)-based filtering mechanism for reliable respiratory estimation. We implement and evaluate the framework on three robotic platforms spanning quadruped and wheeled locomotion and multiple edge-computing architectures. Experiments conducted across diverse lighting conditions, subject poses, and robot-to-subject distances demonstrate that the framework generalizes across platforms without per-platform algorithmic retuning, while revealing modality-specific operational boundaries. RGB provides the broadest coverage up to 8m, NIR remains effective up to 6m, thermal is reliable only at short range, and low-light sensing supports monitoring in complete darkness up to 8m. Overall, the results demonstrate the feasibility of multimodal contactless RR monitoring on mobile robots and support its use as a foundation for autonomous triage and victim assessment in hazardous search-and-rescue settings.

Problem

• contactless respiratory monitoring
• heterogeneous mobile robots
• emergency response
• multimodal sensing
• edge computing

Innovation

• multimodal sensing
• edge computing
• contactless respiratory monitoring
• modality-adaptive
• mobile robotics