LSM-2: Learning from Incomplete Wearable Sensor Data

2025-06-05

AI Summary

Wearable sensor data frequently exhibit high proportions of non-random missingness, severely undermining self-supervised learning (SSL) performance. To address this, we propose LSM-2, a novel SSL framework that eliminates the need for imputation. LSM-2 introduces Adaptive Inheritance-based Masking (AIM), the first mechanism to jointly model natural missingness and artificial masking—enabling explicit awareness of real-world missing patterns and robust representation learning. It incorporates learnable mask tokens to support multimodal temporal modeling and is pretrained at scale on 40 million hours of real-world sensor data. Evaluated across classification, regression, and generative tasks, LSM-2 consistently outperforms state-of-the-art methods. Notably, it maintains strong generalization and inference consistency under high missingness rates and clinically relevant scenarios—e.g., nighttime signal-based hypertension prediction—thereby substantially enhancing clinical deployability and reliability.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Search and Optimization: Learning to Search

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Foundation models, a cornerstone of recent advancements in machine learning, have predominantly thrived on complete and well-structured data. Wearable sensor data frequently suffers from significant missingness, posing a substantial challenge for self-supervised learning (SSL) models that typically assume complete data inputs. This paper introduces the second generation of Large Sensor Model (LSM-2) with Adaptive and Inherited Masking (AIM), a novel SSL approach that learns robust representations directly from incomplete data without requiring explicit imputation. AIM's core novelty lies in its use of learnable mask tokens to model both existing ("inherited") and artificially introduced missingness, enabling it to robustly handle fragmented real-world data during inference. Pre-trained on an extensive dataset of 40M hours of day-long multimodal sensor data, our LSM-2 with AIM achieves the best performance across a diverse range of tasks, including classification, regression and generative modeling. Furthermore, LSM-2 with AIM exhibits superior scaling performance, and critically, maintains high performance even under targeted missingness scenarios, reflecting clinically coherent patterns, such as the diagnostic value of nighttime biosignals for hypertension prediction. This makes AIM a more reliable choice for real-world wearable data applications.

Problem

• Handling incomplete wearable sensor data for self-supervised learning
• Learning robust representations without explicit data imputation
• Maintaining performance under clinically coherent missingness patterns

Innovation

• Adaptive and Inherited Masking for incomplete data
• Learnable mask tokens model missingness
• Pre-trained on 40M hours sensor data