Dynamic Uncertainty-aware Multimodal Fusion for Outdoor Health Monitoring

2025-08-12

AI Summary

In outdoor dynamic environments, health monitoring suffers from low robustness due to severe sensor noise, strong physiological signal fluctuations, and frequent multimodal data missing. To address this, we propose DUAL-Health, an uncertainty-aware multimodal fusion framework. Its core innovations include: (1) the first introduction of a dynamic uncertainty quantification mechanism that jointly assesses input noise and signal volatility via temporal and instantaneous features; (2) an uncertainty-calibrated adaptive modality weighting strategy, coupled with cross-modal distribution alignment in a shared semantic space. Built upon a multimodal large language model (MLLM), DUAL-Health integrates an uncertainty estimation network, an adaptive fusion module, and spatiotemporal alignment mechanisms to enable noise-aware dynamic modeling and missing-data recovery. Experiments demonstrate that DUAL-Health significantly outperforms state-of-the-art methods under high-noise and multimodal missing conditions, achieving substantial improvements in both detection accuracy and robustness.

Technology Category

• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Machine Learning: Multimodal Learning
• Computer Vision: Multi-modal Vision

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG

Abstract

Outdoor health monitoring is essential to detect early abnormal health status for safeguarding human health and safety. Conventional outdoor monitoring relies on static multimodal deep learning frameworks, which requires extensive data training from scratch and fails to capture subtle health status changes. Multimodal large language models (MLLMs) emerge as a promising alternative, utilizing only small datasets to fine-tune pre-trained information-rich models for enabling powerful health status monitoring. Unfortunately, MLLM-based outdoor health monitoring also faces significant challenges: I) sensor data contains input noise stemming from sensor data acquisition and fluctuation noise caused by sudden changes in physiological signals due to dynamic outdoor environments, thus degrading the training performance; ii) current transformer based MLLMs struggle to achieve robust multimodal fusion, as they lack a design for fusing the noisy modality; iii) modalities with varying noise levels hinder accurate recovery of missing data from fluctuating distributions. To combat these challenges, we propose an uncertainty-aware multimodal fusion framework, named DUAL-Health, for outdoor health monitoring in dynamic and noisy environments. First, to assess the impact of noise, we accurately quantify modality uncertainty caused by input and fluctuation noise with current and temporal features. Second, to empower efficient muitimodal fusion with low-quality modalities,we customize the fusion weight for each modality based on quantified and calibrated uncertainty. Third, to enhance data recovery from fluctuating noisy modalities, we align modality distributions within a common semantic space. Extensive experiments demonstrate that our DUAL-Health outperforms state-of-the-art baselines in detection accuracy and robustness.

Problem

• Addressing noise in sensor data for health monitoring
• Improving multimodal fusion in noisy outdoor environments
• Enhancing data recovery from fluctuating noisy modalities

Innovation

• Quantify modality uncertainty using current and temporal features
• Customize fusion weights based on calibrated uncertainty
• Align modality distributions in common semantic space