🤖 AI Summary
This study investigates the potential of body-conducted speech (BCS) sensing for privacy-enhancing wearable interaction. To address the lack of standardized, GDPR-compliant BCS benchmarks, we introduce Vibravox—the first multilingual, multimodal, GDPR-aligned French BCS dataset—comprising 45 hours of synchronized recordings from 188 speakers in a 3D higher-order Ambisonics acoustic field. We systematically integrate five BCS sensor types—including ear-canal microphones and bone/throat-conduction transducers—and co-register each with air-conducted reference speech. Leveraging the rich spatial and modal diversity, we establish comprehensive benchmarks for automatic speech recognition, speech enhancement, and speaker verification using state-of-the-art deep learning models. Experimental results reveal fundamental trade-offs across sensors in noise robustness, phonetic fidelity, and speaker discriminability, providing empirical grounding for cross-modal speech modeling. All data and baseline code are publicly released, filling a critical gap in standardized BCS evaluation.
📝 Abstract
Vibravox is a dataset compliant with the General Data Protection Regulation (GDPR) containing audio recordings using five different body-conduction audio sensors : two in-ear microphones, two bone conduction vibration pickups and a laryngophone. The dataset also includes audio data from an airborne microphone used as a reference. The Vibravox corpus contains 45 hours of speech samples and physiological sounds recorded by 188 participants under different acoustic conditions imposed by an high order ambisonics 3D spatializer. Annotations about the recording conditions and linguistic transcriptions are also included in the corpus. We conducted a series of experiments on various speech-related tasks, including speech recognition, speech enhancement and speaker verification. These experiments were carried out using state-of-the-art models to evaluate and compare their performances on signals captured by the different audio sensors offered by the Vibravox dataset, with the aim of gaining a better grasp of their individual characteristics.