Problem
Research questions and friction points this paper is trying to address.
Pulse Density Modulation
State Space Models
Low-Power Devices
Sampling Rate Invariance
Edge Computing
Multirate State Space Models for End-to-End Processing of Pulse Density Modulated Speech Signals
🤖 AI Summary
本文提出了一种基于状态空间模型的端到端脉冲密度调制语音信号处理架构，解决了低功耗设备上直接处理PDM信号的问题，并能大幅度减少下游处理时间步。
📝 Abstract
Deep neural networks (DNNs) based on state-space models (SSMs) are increasingly applied to speech processing, but typically operate on pulse-code-modulated (PCM) audio. This constrains deployment on low-power, always-on edge devices, which commonly use single-bit pulse-density-modulated (PDM) micro-electromechanical (MEMS) microphones for their noise robustness, low cost, and variable sampling rates that enable low-power operation. In fact, converting PDM to PCM requires low-pass filtering and decimation, imposing costly overhead on resource-constrained hardware. While prior works have attempted to process PDM signals directly, they require long training times and generalize poorly across sampling rates. In this paper, we show that the SSM has two key properties that remediate these issues: its continuous-time parametrization allows it to produce a consistent representation of the input audio signal, regardless of the modulation strategy and sampling rate, and its long-term memory enables this representation to be aggressively downsampled without needing any anti-aliasing operations. We then propose a novel end-to-end PDM speech processing architecture that uses an SSM to encode the input audio signal into a modulation- and sampling-rate-invariant latent representation. We show that our proposed architecture achieves robust speech classification and enhancement gains at low-power sampling-rates (512 kHz) and similar performance to state-of-the-art algorithms operating on PCM data when tested on standard PDM sampling-rates of 2 MHz. Moreover, we show that the SSM's output can be downsampled by more than 65,000 times, thus significantly reducing the number of processing timesteps in downstream layers.
Innovation
Methods, ideas, or system contributions that make the work stand out.
state-space models
pulse-density modulation
end-to-end processing
continuous-time parametrization
downsampling
🔎 Similar Papers
2024-10-01arXiv.orgCitations: 2
