Adaptive Oscillatory Inductive Bias for Modeling Sharp Prosodic Dynamics in Diffusion-Based TTS

2026-06-24

AI Summary

Diffusion-based TTS systems exhibit limitations in modeling sharp prosodic transitions and rapid pitch variations characteristic of expressive speech. To address this, this work proposes OscillaTTS, which introduces an adaptive oscillatory nonlinear activation function within the diffusion decoder, complemented by a linear bypass mechanism. This design enables efficient modeling of abrupt changes in amplitude and frequency while preserving signal stability. The proposed approach substantially enhances the system’s capacity to capture pronounced prosodic dynamics, yielding consistent improvements over baseline methods on both objective metrics and subjective evaluations across the LJSpeech and emotional speech datasets, thereby demonstrating its effectiveness in generating highly expressive synthetic speech.

Abstract

Diffusion-based text-to-speech (TTS) models have achieved significant improvements in speech quality. However, modeling sharp prosodic transitions and rapid pitch variations in expressive speech remains challenging. Existing diffusion-based TTS decoders commonly utilize periodic nonlinearities such as Snake activation function to capture harmonic structures, but this activation funcation provides limited adaptability when modeling abrupt amplitude and frequency variations. In this paper, we investigate the role of oscillatory inductive bias in diffusion-based TTS decoders and introduce an adaptive oscillatory nonlinearity that enables controllable periodic modulation while maintaining signal stability through a linear bypass component. We refer the resulting TTS system as OscillaTTS. Experiments on the LJSpeech and Emotional Speech Dataset show consistent improvements across objective and subjective evaluations, indicating improved modeling of expressive prosodic dynamics.

Problem

• diffusion-based TTS
• prosodic dynamics
• pitch variation
• expressive speech
• oscillatory inductive bias

Innovation

• adaptive oscillatory nonlinearity
• diffusion-based TTS
• prosodic dynamics
• inductive bias
• expressive speech synthesis