PI-HMR: Towards Robust In-bed Temporal Human Shape Reconstruction with Contact Pressure Sensing

2025-02-27

AI Summary

To address the failure of vision-based 3D human reconstruction in unlit, privacy-sensitive bedridden scenarios, this paper proposes a non-line-of-sight, time-series 3D human reconstruction method leveraging pressure-sensing bed sheets. We introduce SMPLify-IB, the first gravity-constrained optimization algorithm tailored for inferring high-fidelity SMPL parameters from pressure distributions under supine postures. Further, we design PI-HMR—a multi-scale spatiotemporal fusion framework integrating temporal convolutions, graph neural networks, and spatial position priors—to explicitly model the coupling between pressure patterns and soft-tissue deformations. Evaluated on our newly established in-bed pressure dataset, our method reduces MPJPE by 17.01 mm over state-of-the-art methods. To our knowledge, this is the first approach achieving simultaneous, high-accuracy (sub-centimeter), real-time (≥30 FPS), and privacy-preserving monitoring of full-body pose and soft-tissue deformation within a bed environment.

Technology Category

• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Computer Vision: 3D Computer Vision
• Humans and AI: Human-Aware Planning and Behavior Prediction

Application Category

• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets

Abstract

Long-term in-bed monitoring benefits automatic and real-time health management within healthcare, and the advancement of human shape reconstruction technologies further enhances the representation and visualization of users' activity patterns. However, existing technologies are primarily based on visual cues, facing serious challenges in non-light-of-sight and privacy-sensitive in-bed scenes. Pressure-sensing bedsheets offer a promising solution for real-time motion reconstruction. Yet, limited exploration in model designs and data have hindered its further development. To tackle these issues, we propose a general framework that bridges gaps in data annotation and model design. Firstly, we introduce SMPLify-IB, an optimization method that overcomes the depth ambiguity issue in top-view scenarios through gravity constraints, enabling generating high-quality 3D human shape annotations for in-bed datasets. Then we present PI-HMR, a temporal-based human shape estimator to regress meshes from pressure sequences. By integrating multi-scale feature fusion with high-pressure distribution and spatial position priors, PI-HMR outperforms SOTA methods with 17.01mm Mean-Per-Joint-Error decrease. This work provides a whole

Problem

• Overcomes depth ambiguity in top-view human shape reconstruction.
• Enhances real-time motion reconstruction using pressure-sensing bedsheets.
• Improves 3D human shape estimation with multi-scale feature fusion.

Innovation

• SMPLify-IB optimizes 3D shape annotations.
• PI-HMR estimates human shape from pressure.
• Multi-scale fusion improves reconstruction accuracy.