FROST-EMA: Finnish and Russian Oral Speech Dataset of Electromagnetic Articulography Measurements with L1, L2 and Imitated L2 Accents

2025-06-10

AI Summary

This study investigates articulatory variability in Finnish–Russian bilinguals across three conditions: native language (L1), second language (L2), and imitated L2 accent. It addresses the impact of accent imitation on speech technology robustness and articulatory physiology. Using electromagnetic articulography (EMA), the first high-temporal-spatial-resolution bilingual EMA corpus—spanning multiple accents and languages—was constructed, with synchronized EMA data collected from the same speakers across all three conditions. Integrating acoustic-articulatory joint analysis and automatic speaker verification (ASV) experiments, results show that imitated L2 significantly degrades ASV system robustness. Crucially, tongue movement trajectories in imitated L2 lack the physiological adaptations characteristic of authentic L2 production, revealing a fundamental dissociation between perceptual imitation and motor-physiological competence. This work establishes a novel paradigm for evaluating cross-accent speech technologies, advances physiological modeling of second-language acquisition, and provides empirical foundations for enhancing robustness in automatic speech recognition.

Technology Category

• Natural Language Processing: Machine Translation, Multilinguality, Cross-Lingual NLP
• Cognitive Modeling & Cognitive Systems: Simulating Human Behavior
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search
• Web Mining and Content Analysis: Mining multimedia, multimodal, multilingual, cross-lingual Web data
• User Modeling, Personalization and Recommendation: User modeling and simulation for interactive and conversational systems

Abstract

We introduce a new FROST-EMA (Finnish and Russian Oral Speech Dataset of Electromagnetic Articulography) corpus. It consists of 18 bilingual speakers, who produced speech in their native language (L1), second language (L2), and imitated L2 (fake foreign accent). The new corpus enables research into language variability from phonetic and technological points of view. Accordingly, we include two preliminary case studies to demonstrate both perspectives. The first case study explores the impact of L2 and imitated L2 on the performance of an automatic speaker verification system, while the second illustrates the articulatory patterns of one speaker in L1, L2, and a fake accent.

Problem

• Analyzing phonetic variability in L1, L2, and imitated L2 speech
• Investigating automatic speaker verification performance across accents
• Examining articulatory patterns in native and non-native speech

Innovation

• Bilingual speech corpus with L1, L2, imitated L2
• Electromagnetic Articulography for articulatory patterns
• Case studies on speaker verification and articulation