ArtNet: A JEPA-Like Articulatory Predictive Framework for Robust Zero-Shot Phoneme Recognition

2026-06-15

AI Summary

This work addresses the vulnerability of zero-shot cross-lingual phoneme recognition to language-specific acoustic variations, which impairs the robustness of acoustic-to-symbol mapping. To mitigate this issue, the authors propose ArtNet, a novel framework that integrates a JEPA-inspired articulatory prediction architecture with self-supervised acoustic features, articulatory representation learning, and a variational information bottleneck to suppress language-irrelevant variability. The core innovation lies in the Vector Space Inventory Alignment (VSIA) strategy, which enables structured articulatory prediction and cross-lingual phonemic alignment. Evaluated on seven unseen languages, the proposed method significantly outperforms existing baselines, achieving a relative reduction of 20.56% in phoneme error rate and 7.01% in phonetic feature error rate.

Abstract

Zero-shot cross-lingual phoneme recognition is often hindered by the fragility of direct acoustic-to-symbol mapping, which is susceptible to language-specific variations. Echoing joint-embedding predictive architecture (JEPA) work in vision, we propose ArtNet, a framework that explores a structured feature prediction task based on articulatory features to enhance acoustic robustness. Specifically, ArtNet integrates an articulatory predictor, designed to extract universal articulatory representations from self-supervised learning (SSL) features, with a variational information bottleneck (VIB) to suppress language-specific variations. Experiments on seven unseen languages demonstrate that ArtNet, particularly when synergized with the proposed vector-space inventory alignment (VSIA) strategy, significantly outperforms competitive baselines, achieving a 20.56\% relative reduction in phoneme error rate (PER) and 7.01\% in phoneme feature error rate (PFER).

Problem

• zero-shot phoneme recognition
• cross-lingual
• acoustic-to-symbol mapping
• language-specific variations
• articulatory features

Innovation

• articulatory features
• zero-shot phoneme recognition
• variational information bottleneck
• self-supervised learning
• vector-space inventory alignment