Problem
Research questions and friction points this paper is trying to address.
low-bitrate
neural speech codec
vector quantization
PACodec: A Low-bitrate Neural Speech Codec with Parallel Additive Vector Quantization
๐ค AI Summary
ๆฌๆๆๅบPACodec๏ผไธ็งๅบไบๅนถ่กๅ ๆง็ข้้ๅ็ๆฐไฝๆฏ็น็็ฅ็ป่ฏญ้ณ็ผ็ ๅจ๏ผ้่ฟไผๅ็ ็ไฝฟ็จๅๅฐ30%ๆฏ็น็๏ผๅๆถไฟๆ่งฃ็ ่ดจ้ใ
๐ Abstract
This paper proposes PACodec, a novel low-bitrate neural speech codec based on parallel additive vector quantization (PAVQ). Unlike the mainstream residual vector quantization (RVQ) used in most neural speech codecs, where vector quantizers (VQs) are sequentially dependent, the PAVQ strategy adopted in PACodec aggregates parallel quantization results to optimize bitrate usage. Specifically, the PAVQ adopts a "global-local-global" (GLG) design: the global encoded features are quantized in parallel by multiple independent VQs, each attending to a local component of the representation, and their outputs are aggregated through addition to yield the final global quantization result for decoding. Experimental results show that PACodec, as each VQ focuses only on local information, supports smaller codebooks and reduces bitrate by 30% compared with baselines at the same decoding quality, with only minor model complexity. Further analysis shows that, owing to the GLG framework of PAVQ, the proposed PACodec is disentanglement-friendly, and each independent VQ captures different aspects of speech, e.g., content, timbre, and acoustic details, suggesting potential for application to downstream tasks such as voice conversion.
Innovation
Methods, ideas, or system contributions that make the work stand out.
Parallel Additive Vector Quantization
Low-bitrate Neural Speech Codec
Global-Local-Global Design
Bitrate Reduction
๐ Similar Papers
No similar papers found.
