SlimSpeech: Lightweight and Efficient Text-to-Speech with Slim Rectified Flow

2025-04-06

IEEE International Conference on Acoustics, Speech, and Signal Processing

AI Summary

To address the inefficiency of streaming text-to-speech (TTS) systems—characterized by large parameter counts, high sampling steps, and slow inference—this paper proposes SlimSpeech, a lightweight, single-step, high-fidelity TTS framework. Our method introduces a compact rectified flow model and a novel Slim Reflow distillation strategy that jointly optimizes trajectory straightness and knowledge transfer to drastically reduce model size. By integrating streamlined network architecture design with fine-grained reflow path adjustment, SlimSpeech achieves speech quality comparable to state-of-the-art streaming flow-based models using only one denoising step. Experiments demonstrate that SlimSpeech reduces model parameters by over 70% and accelerates inference by 3–5×, while preserving naturalness and intelligibility. This work establishes a new paradigm for efficient, edge-deployable TTS systems.

Technology Category

• Natural Language Processing: Speech
• Machine Learning: Learning on the Edge & Model Compression
• Planning, Routing, and Scheduling: Planning with Language Models

Application Category

• Search and Retrieval-Augmented AI: Efficiency and scalability of Web search engines
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Systems and Infrastructure for Web, Mobile and WoT: Data management and stream processing for Web, mobile and wireless applications

Abstract

Recently, flow matching based speech synthesis has significantly enhanced the quality of synthesized speech while reducing the number of inference steps. In this paper, we introduce SlimSpeech, a lightweight and efficient speech synthesis system based on rectified flow. We have built upon the existing speech synthesis method utilizing the rectified flow model, modifying its structure to reduce parameters and serve as a teacher model. By refining the reflow operation, we directly derive a smaller model with a more straight sampling trajectory from the larger model, while utilizing distillation techniques to further enhance the model performance. Experimental results demonstrate that our proposed method, with significantly reduced model parameters, achieves comparable performance to larger models through one-step sampling.

Problem

• Lightweight text-to-speech with reduced parameters
• Efficient speech synthesis via rectified flow
• One-step sampling performance comparable to larger models

Innovation

• Lightweight rectified flow for speech synthesis
• Refined reflow operation for smaller model
• Distillation enhances performance with fewer parameters