GrainSpeech: Less Context, More Detail for Compact Speech Synthesis

2026-09-16

AI Summary

研究通过减少编码器上下文和改进Mel谱图监督方法，解决了紧凑声学模型的质量-容量权衡问题，提出GrainSpeech模型。

Abstract

Compact acoustic models face a challenging quality-capacity trade-off. We investigate two factors in this regime: encoder context and Mel-spectrogram supervision. A receptive-field-scaling study shows that expanding self-attention beyond 15 phonemes provides no consistent gains in pitch, energy, or duration prediction. Guided by this finding, we introduce a fixed-receptive-field convolutional encoder that reduces the respective prediction errors by 36.0%, 17.3%, and 3.4%. We further show that directly transferring image-domain gradient-variance supervision restores fine-scale variation but degrades predicted quality, motivating a Mel-specific formulation with axis-specific gradients, overlapping local statistics, and log-domain variance matching. GrainSpeech contains only 264.8K parameters and achieves 17.9x real-time Mel generation on a microcontroller (MCU), while attaining UTMOS scores comparable to substantially larger models with less than 1.5% of their parameters. Source code and demos are available at https://github.com/lab-emi/GrainSpeech.

Problem

• Compact acoustic models
• encoder context
• Mel-spectrogram supervision
• quality-capacity trade-off

Innovation

• fixed-receptive-field convolutional encoder
• gradient-variance supervision
• compact speech synthesis