Bolbosh: Script-Aware Flow Matching for Kashmiri Text-to-Speech

2026-03-08

AI Summary

This work addresses the absence of dedicated text-to-speech (TTS) systems for Kashmiri, a low-resource language whose Perso-Arabic script—characterized by complex diacritics and intricate phonological structure—severely degrades the performance of generic multilingual models. We present the first open-source neural TTS system specifically designed for Kashmiri, which innovatively integrates script-aware modeling with Optimal Transport Conditional Flow Matching (OT-CFM) to explicitly capture diacritic and vowel nuances. To mitigate data heterogeneity, we apply a three-stage acoustic enhancement pipeline comprising dereverberation, silence trimming, and loudness normalization. Despite limited resources, our system achieves stable alignment and high-quality synthesis, attaining a Mean Opinion Score (MOS) of 3.63—significantly outperforming a zero-shot baseline (1.86)—and a Mel-Cepstral Distortion (MCD) of 3.73, thereby establishing a new benchmark for Kashmiri TTS.

Technology Category

• Natural Language Processing: Speech
• Data Mining & Knowledge Management: Conversational Systems for Recommendation & Retrieval
• Machine Learning: Large Multimodal Models (LMMs)

Application Category

• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search
• Web Mining and Content Analysis: Mining multimedia, multimodal, multilingual, cross-lingual Web data
• Systems and Infrastructure for Web, Mobile and WoT: Experiences and lessons learnt from Web-based algorithms and system deployments

Abstract

Kashmiri is spoken by around 7 million people but remains critically underserved in speech technology, despite its official status and rich linguistic heritage. The lack of robust Text-to-Speech (TTS) systems limits digital accessibility and inclusive human-computer interaction for native speakers. In this work, we present the first dedicated open-source neural TTS system designed for Kashmiri. We show that zero-shot multilingual baselines trained for Indic languages fail to produce intelligible speech, achieving a Mean Opinion Score (MOS) of only 1.86, largely due to inadequate modeling of Perso-Arabic diacritics and language-specific phonotactics. To address these limitations, we propose Bolbosh, a supervised cross-lingual adaptation strategy based on Optimal Transport Conditional Flow Matching (OT-CFM) within the Matcha-TTS framework. This enables stable alignment under limited paired data. We further introduce a three-stage acoustic enhancement pipeline consisting of dereverberation, silence trimming, and loudness normalization to unify heterogeneous speech sources and stabilize alignment learning. The model vocabulary is expanded to explicitly encode Kashmiri graphemes, preserving fine-grained vowel distinctions. Our system achieves a MOS of 3.63 and a Mel-Cepstral Distortion (MCD) of 3.73, substantially outperforming multilingual baselines and establishing a new benchmark for Kashmiri speech synthesis. Our results demonstrate that script-aware and supervised flow-based adaptation are critical for low-resource TTS in diacritic-sensitive languages. Code and data are available at: https://github.com/gaash-lab/Bolbosh.

Problem

• Kashmiri
• Text-to-Speech
• low-resource
• diacritic-sensitive
• speech synthesis

Innovation

• script-aware TTS
• Optimal Transport Conditional Flow Matching
• low-resource speech synthesis
• acoustic enhancement pipeline
• cross-lingual adaptation