ARCHI-TTS: A flow-matching-based Text-to-Speech Model with Self-supervised Semantic Aligner and Accelerated Inference

2026-02-05

AI Summary

This work addresses the challenges of alignment modeling and high inference cost in non-autoregressive text-to-speech (TTS) synthesis by proposing a flow-matching-based TTS model. The approach introduces a self-supervised semantic aligner to effectively enhance temporal and semantic consistency between text and audio, while incorporating an encoder feature reuse mechanism to accelerate inference. Additionally, a CTC auxiliary loss is integrated to strengthen semantic modeling. Experimental results demonstrate that the proposed model achieves competitive performance, attaining a word error rate (WER) of 1.98% on LibriSpeech-PC test-clean and 1.47% and 1.42% on SeedTTS test-en and test-zh, respectively. The system exhibits both high inference efficiency and superior performance compared to current state-of-the-art methods.

Technology Category

• Natural Language Processing: Sentence-level Semantics, Textual Inference, etc.
• Machine Learning: Large Multimodal Models (LMMs)
• Planning, Routing, and Scheduling: Planning with Language Models

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Large language models for search
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

Although diffusion-based, non-autoregressive text-to-speech (TTS) systems have demonstrated impressive zero-shot synthesis capabilities, their efficacy is still hindered by two key challenges: the difficulty of text-speech alignment modeling and the high computational overhead of the iterative denoising process. To address these limitations, we propose ARCHI-TTS that features a dedicated semantic aligner to ensure robust temporal and semantic consistency between text and audio. To overcome high computational inference costs, ARCHI-TTS employs an efficient inference strategy that reuses encoder features across denoising steps, drastically accelerating synthesis without performance degradation. An auxiliary CTC loss applied to the condition encoder further enhances the semantic understanding. Experimental results demonstrate that ARCHI-TTS achieves a WER of 1.98% on LibriSpeech-PC test-clean, and 1.47%/1.42% on SeedTTS test-en/test-zh with a high inference efficiency, consistently outperforming recent state-of-the-art TTS systems.

Problem

• text-to-speech
• alignment modeling
• computational overhead
• non-autoregressive TTS
• zero-shot synthesis

Innovation

• flow-matching
• semantic aligner
• accelerated inference
• non-autoregressive TTS
• self-supervised alignment