AURORA Model of Formant-to-Tongue Inversion for Didactic and Clinical Applications

2026-03-18

AI Summary

This study addresses the inverse problem of inferring tongue shape and displacement from the first two formant frequencies (F1 and F2) of vowels, with applications in phonetic pedagogy and clinical speech therapy. Leveraging synchronized ultrasound tongue imaging and acoustic data from 40 native English speakers, the work proposes the first machine learning model that maps formant values to articulatory tongue morphology. The system integrates real-time biofeedback and interactive visualization capabilities, implemented through two user-friendly tools: a Shiny web application and a biofeedback prototype. These tools enable qualitatively accurate prediction of key tongue features, offering an innovative technological framework to support both speech science education and therapeutic interventions for speech disorders.

Technology Category

• Natural Language Processing: Speech
• Machine Learning: Learning with Manifolds
• Humans and AI: Interaction Techniques and Devices

Application Category

• User Modeling, Personalization and Recommendation: User modeling and simulation for interactive and conversational systems
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search

Abstract

This paper outlines the conceptual and computational foundations of the AURORA (Acoustic Understanding and Real-time Observation of Resonant Articulations) model. AURORA predicts tongue displacement and shape in vowel sounds based on the first two formant values. It is intended as a didactic aid helping to explain the relationship between formants and the underlying articulation, as well as a foundation for biofeedback applications. The model is informed by ultrasound tongue imaging and acoustic data from 40 native speakers of English. In this paper we discuss the motivation for the model, the modelling objectives as well as the model architecture. We provide a qualitative evaluation of the model, focusing on selected tongue features. We then present two tools developed to make the model more accessible to a wider audience, a Shiny app and a prototype software for real-time tongue biofeedback. Potential users include students of phonetics, linguists in fields adjacent to phonetics, as well as speech and language therapy practitioners and clients.

Problem

• formant-to-tongue inversion
• tongue displacement
• vowel articulation
• biofeedback
• speech acoustics

Innovation

• formant-to-tongue inversion
• ultrasound tongue imaging
• real-time biofeedback
• articulatory modeling
• speech didactics