V(is)owel: An Interactive Vowel Chart to Understand What Makes Visual Pronunciation Effective in Second Language Learning

2025-07-08

AI Summary

The mechanisms underlying the effectiveness of visual feedback in second-language (L2) pronunciation learning remain poorly understood. To address this, we designed V(is)owel—an interactive, anatomy-informed vowel visualization tool—that maps real-time tongue position and movement directly onto a dynamic vowel chart, enabling synchronized audiovisual feedback. Unlike conventional abstract representations (e.g., waveforms or spectrograms), V(is)owel provides an embodied, anatomically grounded mapping between articulatory motion and visual output, enhancing perceptual accessibility and practice motivation for non-specialist learners. Integrating real-time speech analysis, intuitive human-computer interaction, and self-directed practice tracking, our controlled experiment demonstrated significantly higher practice frequency among V(is)owel users compared to an auditory-only control group. These results validate its efficacy in improving feedback operability, supporting autonomous learning, and strengthening articulatory motor representations. The study elucidates the cognitive mechanisms by which embodied visual feedback facilitates L2 phonetic acquisition and establishes a novel design paradigm for pronunciation instruction technologies.

Technology Category

• Computer Vision: Language and Vision
• Natural Language Processing: Language Grounding & Multi-modal NLP
• Humans and AI: Interaction Techniques and Devices

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Metrics for user behavior and evaluating success
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

Visual feedback speeds up learners' improvement of pronunciation in a second language. The visual combined with audio allows speakers to see sounds and differences in pronunciation that they are unable to hear. Prior studies have tested different visual methods for improving pronunciation, however, we do not have conclusive understanding of what aspects of the visualizations contributed to improvements. Based on previous work, we created V(is)owel, an interactive vowel chart. Vowel charts provide actionable feedback by directly mapping physical tongue movement onto a chart. We compared V(is)owel with an auditory-only method to explore how learners parse visual and auditory feedback to understand how and why visual feedback is effective for pronunciation improvement. The findings suggest that designers should include explicit anatomical feedback that directly maps onto physical movement for phonetically untrained learners. Furthermore, visual feedback has the potential to motivate more practice since all eight of the participants cited using the visuals as a goal with V(is)owel versus relying on their own judgment with audio alone. Their statements are backed up by all participants practicing words with V(is)owel more than with audio-only. Our results indicate that V(is)owel is effective at providing actionable feedback, demonstrating the potential of visual feedback methods in second language learning.

Problem

• Understanding effective visual feedback for pronunciation improvement
• Comparing visual and auditory feedback in language learning
• Designing actionable visual feedback for phonetically untrained learners

Innovation

• Interactive vowel chart for visual pronunciation feedback
• Direct mapping of tongue movement onto visual chart
• Combined visual and auditory feedback enhances learning