🤖 AI Summary
This work addresses the challenge of achieving both computational efficiency and high audio fidelity in low-precision text-to-speech (TTS) inference, which traditionally suffers from audible distortions and spectral artifacts. The authors propose a precision-aware architecture co-designed with hardware-software optimizations, enabling the first production-grade TTS system on the Tenstorrent platform to deploy 95% low-fidelity computations and 80% BlockFloat8 operations without perceptible quality degradation. By integrating BlockFloat8 quantization, an on-chip network (NoC), distributed SRAM, and a deterministic execution model, the approach substantially reduces memory traffic and computational overhead. Compared to an NVIDIA L40S GPU, the solution achieves comparable throughput at approximately one-fourth the accelerator cost while preserving high-fidelity audio output, thereby redefining the economics of real-time speech synthesis.
📝 Abstract
Text-to-Speech (TTS) models are significantly more numerically fragile than Large Language Models (LLMs) due to their continuous waveform generation and perceptual sensitivity to small numerical perturbations. While aggressive precision reduction techniques such as BlockFloat8 (BFP8) and low-fidelity (LoFi) compute have been widely adopted in language models, applying similar strategies to TTS systems often results in audible artifacts, phase instability, and spectral distortion. In this work, we present Lightning V2, a production-grade TTS model co-optimized for Tenstorrent hardware. Through precision-aware architectural design and hardware-software co-optimization, we achieve over 95% LoFi computational fidelity and more than 80% BlockFloat8 deployment without measurable degradation in audio quality. Leveraging Tenstorrent's Network-on-Chip (NoC), distributed SRAM, and deterministic execution model, we reduce memory movement and redundant weight fetches, enabling efficient low-precision inference. Compared to an NVIDIA L40S baseline, Lightning V2 achieves approximately 4x lower on-prem accelerator cost at equivalent throughput, while maintaining production audio fidelity. Our results demonstrate that precision co-design, combined with hardware-aware optimization, can fundamentally reshape the economics of real-time speech inference.