Phone Segmentation and Recognition through Phonological Activation Mapping

2026-07-09

AI Summary

This work proposes a unified framework for phoneme segmentation and recognition based on a self-supervised speech model (S3M), addressing the limitation of existing approaches that model these tasks independently and thus hinder joint optimization. The method introduces a Phonological Activation Mapping (SPAM) module that transforms frame-level representations into vectors of phonological feature activations—such as voicing and nasality—and employs a lightweight, gradient-free prediction head built atop these features to simultaneously perform segmentation and recognition. Remarkably, the model requires less than one minute of phoneme-level annotations for training, demonstrates strong generalization to unseen phonemes, and achieves state-of-the-art performance across multiple benchmark datasets.

Abstract

Phone segmentation and recognition are inherently related tasks, yet modern approaches typically model them separately. We argue that phonetic structure is already latent in the representations of self-supervised speech models (S3Ms), and one only needs to steer them to solve both tasks. We leverage S3M-based Phonological Activation Mapping (SPAM), which maps each S3M representation frame to a vector of phonological feature activations, such as voicing and nasality. On top of SPAM, we introduce two simple but effective lightweight, gradient-descent-free prediction heads: a recognition head and a segmentation head. Our method requires less than a minute of phonetic transcriptions, and generalizes to unseen phones during training. Across a diverse range of datasets, our approach attains strong segmentation and recognition performance.

Problem

• phone segmentation
• phone recognition
• phonological features
• self-supervised speech models

Innovation

• self-supervised speech models
• phonological activation mapping
• phone segmentation
• phone recognition
• lightweight prediction heads