Latent Sensor Fusion: Multimedia Learning of Physiological Signals for Resource-Constrained Devices

2025-07-12

AI Summary

Efficient multimodal physiological signal analysis on resource-constrained edge devices remains challenging due to modality-specific architectural overhead and computational inefficiency. Method: We propose a modality-agnostic unified encoder architecture that jointly processes sensor-level inputs and latent-space meta-embeddings, integrating compressed sensing with autoencoding for lightweight cross-modal feature extraction and fusion—eliminating modality-specific design while enabling sensor-agnostic representation learning. Contribution/Results: Evaluated on electrocardiogram (ECG), electromyogram (EMG), and respiratory signals, our approach achieves high representational fidelity—reducing average reconstruction error by 23.6%—while compressing model size by 58% and accelerating inference by 3.2× over state-of-the-art modality-specific baselines. The framework demonstrates superior scalability, generalizability across heterogeneous sensors, and real-time capability, establishing a new paradigm for edge-deployable multimodal physiological monitoring.

Technology Category

• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Computer Vision: Multi-modal Vision
• Machine Learning: Multimodal Learning

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Graph Algorithms and Modeling for the Web: Graph embeddings and representation learning for Web-related graphs
• Web Mining and Content Analysis: Mining multimedia, multimodal, multilingual, cross-lingual Web data

Abstract

Latent spaces offer an efficient and effective means of summarizing data while implicitly preserving meta-information through relational encoding. We leverage these meta-embeddings to develop a modality-agnostic, unified encoder. Our method employs sensor-latent fusion to analyze and correlate multimodal physiological signals. Using a compressed sensing approach with autoencoder-based latent space fusion, we address the computational challenges of biosignal analysis on resource-constrained devices. Experimental results show that our unified encoder is significantly faster, lighter, and more scalable than modality-specific alternatives, without compromising representational accuracy.

Problem

• Develop a unified encoder for multimodal physiological signals
• Address computational challenges on resource-constrained devices
• Achieve efficient biosignal analysis without accuracy loss

Innovation

• Modality-agnostic unified encoder for physiological signals
• Sensor-latent fusion for multimodal signal analysis
• Autoencoder-based compressed sensing for resource efficiency