Beyond Hearing: Learning Task-agnostic ExG Representations from Earphones via Physiology-informed Tokenization

2025-10-22

AI Summary

To address two key bottlenecks in ear-worn electrophysiological (ExG) signal modeling—data scarcity (largely due to controlled lab settings) and poor model generalizability (stemming from task-specific architectures)—this paper introduces the first foundational model for ear-worn ExG signals tailored to free-living scenarios. We develop a custom ear-worn hardware platform to collect the 50-hour multimodal DailySense dataset under naturalistic conditions. We propose Physiological-information-guided Multi-band Tokenization (PiMT), a novel tokenization method that enables cross-task, adaptive full-spectrum representation learning by leveraging physiological priors. PiMT is integrated into an autoencoder-based self-supervised pretraining framework. Extensive evaluation on DailySense and four public benchmarks demonstrates its effectiveness. Results show PiMT consistently outperforms state-of-the-art methods across five sensory tasks, significantly enhancing representation robustness and multi-task generalization capability.

Technology Category

• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Machine Learning: Multimodal Learning
• Humans and AI: Brain-Sensing and Analysis

Application Category

• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Economics, Online Markets and Human Computation: Trust and reliance of crowd workers and data experts on GenAI
• Web Mining and Content Analysis: Large pretrained models with web data

Abstract

Electrophysiological (ExG) signals offer valuable insights into human physiology, yet building foundation models that generalize across everyday tasks remains challenging due to two key limitations: (i) insufficient data diversity, as most ExG recordings are collected in controlled labs with bulky, expensive devices; and (ii) task-specific model designs that require tailored processing (i.e., targeted frequency filters) and architectures, which limit generalization across tasks. To address these challenges, we introduce an approach for scalable, task-agnostic ExG monitoring in the wild. We collected 50 hours of unobtrusive free-living ExG data with an earphone-based hardware prototype to narrow the data diversity gap. At the core of our approach is Physiology-informed Multi-band Tokenization (PiMT), which decomposes ExG signals into 12 physiology-informed tokens, followed by a reconstruction task to learn robust representations. This enables adaptive feature recognition across the full frequency spectrum while capturing task-relevant information. Experiments on our new DailySense dataset-the first to enable ExG-based analysis across five human senses-together with four public ExG benchmarks, demonstrate that PiMT consistently outperforms state-of-the-art methods across diverse tasks.

Problem

• Developing task-agnostic ExG monitoring for real-world applications
• Addressing limited data diversity in electrophysiological signal collection
• Overcoming task-specific model limitations for cross-task generalization

Innovation

• Earphone-based hardware collects free-living ExG data
• Physiology-informed tokenization creates 12 signal tokens
• Reconstruction task learns robust task-agnostic representations