HAR-DoReMi: Optimizing Data Mixture for Self-Supervised Human Activity Recognition Across Heterogeneous IMU Datasets

2025-03-16

AI Summary

Weak generalization across heterogeneous IMU datasets remains a key challenge in human activity recognition (HAR). To address this, we propose a self-supervised pre-training framework tailored for continuous multi-channel sensor signals. Our method introduces two core innovations: (1) the first DoReMi variant adapted to IMU data, replacing NLL-based language modeling with MSE-based masked signal reconstruction; and (2) integration of the Mahony attitude estimation algorithm to align coordinate systems across devices, thereby mitigating sensor orientation heterogeneity. Evaluated on multiple cross-dataset transfer tasks, our approach achieves an average accuracy improvement of 6.51% over prior methods. Moreover, it attains superior performance using only 30–50% of the labeled data required by state-of-the-art approaches, significantly enhancing model practicality and deployment efficiency.

Technology Category

• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Machine Learning: Learning with Manifolds
• Planning, Routing, and Scheduling: Activity and Plan Recognition

Application Category

• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs

Abstract

Cross-dataset Human Activity Recognition (HAR) suffers from limited model generalization, hindering its practical deployment. To address this critical challenge, inspired by the success of DoReMi in Large Language Models (LLMs), we introduce a data mixture optimization strategy for pre-training HAR models, aiming to improve the recognition performance across heterogeneous datasets. However, directly applying DoReMi to the HAR field encounters new challenges due to the continuous, multi-channel and intrinsic heterogeneous characteristics of IMU sensor data. To overcome these limitations, we propose a novel framework HAR-DoReMi, which introduces a masked reconstruction task based on Mean Squared Error (MSE) loss. By raplacing the discrete language sequence prediction task, which relies on the Negative Log-Likelihood (NLL) loss, in the original DoReMi framework, the proposed framework is inherently more appropriate for handling the continuous and multi-channel characteristics of IMU data. In addition, HAR-DoReMi integrates the Mahony fusion algorithm into the self-supervised HAR pre-training, aiming to mitigate the heterogeneity of varying sensor orientation. This is achieved by estimating the sensor orientation within each dataset and facilitating alignment with a unified coordinate system, thereby improving the cross-dataset generalization ability of the HAR model. Experimental evaluation on multiple cross-dataset HAR transfer tasks demonstrates that HAR-DoReMi improves the accuracy by an average of 6.51%, compared to the current state-of-the-art method with only approximately 30% to 50% of the data usage. These results confirm the effectiveness of HAR-DoReMi in improving the generalization and data efficiency of pre-training HAR models, underscoring its significant potential to facilitate the practical deployment of HAR technology.

Problem

• Improves cross-dataset generalization for Human Activity Recognition.
• Optimizes data mixture for self-supervised HAR using IMU datasets.
• Addresses heterogeneity in sensor data for better model accuracy.

Innovation

• Optimizes data mixture for HAR model pre-training.
• Uses MSE loss for masked reconstruction task.
• Integrates Mahony fusion for sensor orientation alignment.