Toward Foundation Model for Multivariate Wearable Sensing of Physiological Signals

2024-12-12

arXiv.org

AI Summary

This work addresses the weak representation generalizability of multimodal wearable physiological signals (PPG, ECG, EEG, GSR, IMU) caused by device heterogeneity, divergent sampling rates, and inconsistent channel configurations. To this end, we propose NormWear—the first universal foundation model for wearable physiological signals. Methodologically, NormWear introduces a channel-aware attention mechanism and a shared [CLS] token to jointly model univariate temporal dynamics and cross-channel couplings, and employs a Transformer-based architecture for multimodal self-supervised pretraining, enabling flexible zero-shot to full-shot transfer. Evaluated across 11 public datasets and 18 health-related tasks—including mental health assessment, vital sign estimation, and disease risk prediction—NormWear consistently outperforms state-of-the-art baselines, demonstrating strong generalization across diverse devices and real-world scenarios.

Technology Category

• Machine Learning: Multimodal Learning
• Computer Vision: Multi-modal Vision
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Systems and Infrastructure for Web, Mobile and WoT: Web applications in cross-disciplinary domains and verticals such as mixed reality, smart cities, and digital health
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for heterogeneous, signed, attributed, multi-relational, temporal, higher-order, and annotated Web-related graphs

Abstract

Time-series foundation models excel at tasks like forecasting across diverse data types by leveraging informative waveform representations. Wearable sensing data, however, pose unique challenges due to their variability in patterns and frequency bands, especially for healthcare-related outcomes. The main obstacle lies in crafting generalizable representations that adapt efficiently across heterogeneous sensing configurations and applications. To address this, we propose NormWear, the first multi-modal and ubiquitous foundation model designed to extract generalized and informative representations from wearable sensing data. Specifically, we design a channel-aware attention mechanism with a shared special liaison [CLS] token to detect signal patterns in both intra-sensor and inter-sensors. This helps the model to extract more meaningful information considering both time series themselves and the relationships between input sensors. This helps the model to be widely compatible with various sensors settings. NormWear is pretrained on a diverse set of physiological signals, including PPG, ECG, EEG, GSR, and IMU, from various public datasets. Our model shows exceptional generalizability across 11 public wearable sensing datasets, spanning 18 applications in mental health, body state inference, vital sign estimation, and disease risk evaluation. It consistently outperforms competitive baselines under zero-shot, partial-shot, and full-shot settings, indicating broad applicability in real-world health applications.

Problem

• Generalizable representations for diverse wearable sensing data
• Adapting across heterogeneous sensing configurations and applications
• Extracting meaningful information from multi-modal physiological signals

Innovation

• Channel-aware attention mechanism for signal patterns
• Multi-modal foundation model for wearable data
• Pretrained on diverse physiological signals datasets