Using embeddings to predict spoken word duration and pitch in Mandarin monosyllabic words

2026-07-02

AI Summary

This study investigates whether contextual embeddings can effectively predict the phonetic duration of monosyllabic CV words in natural Mandarin speech and enable high-fidelity reconstruction of fundamental frequency (F0) contours. Integrating contextual embeddings with a duration regression model, temporal denormalization, and a permutation-based baseline, this work provides the first evidence—both at the word-type and token levels—that contextual embeddings significantly predict articulatory duration. Experimental results demonstrate that the proposed approach substantially outperforms random baselines, yielding reconstructed F0 contours that closely approximate ground-truth speech at millisecond-level temporal resolution. This achievement establishes an accurate mapping from dimensionless normalized time to actual phonetic duration, thereby bridging abstract linguistic representations with fine-grained prosodic realization.

Abstract

Time-normalized f0 contours of Mandarin words in conversational speech have been shown to be predictable in part from their contextualized embeddings (CEs). The present study investigates whether CEs also predict spoken word duration for 7470 tokens of Mandarin monosyllabic CV words extracted from a Mandarin corpus of spontaneous speech. We show that CEs indeed are predictive for duration, above chance level, not only at the type level, but also at the level of individual tokens, as indicated by the results obtained with the type-wise and token-wise permutation baselines. We also show that the predicted durations are sufficiently precise to back-transform predicted f0 contours in [0,1] normalized time to contours on the ms time scale. The resulting predicted contours approximate empirical contours and also outperform a permutation baseline.

Problem

• spoken word duration
• pitch prediction
• Mandarin monosyllabic words
• contextualized embeddings
• f0 contours

Innovation

• contextualized embeddings
• word duration prediction
• Mandarin prosody
• f0 contour modeling
• spontaneous speech