How Well Do Self-Supervised Speech Models Encode Age and Gender in Children's Speech? A Layer-Wise Analysis Across Multiple Architectures

2026-06-20

AI Summary

This study addresses the challenges of modeling child speech—characterized by high pitch, substantial pronunciation variability, and rapidly evolving acoustic properties with age—by systematically evaluating the encoding capacity of four self-supervised speech models (Wav2Vec2, HuBERT, Data2Vec, and WavLM) for speaker attributes such as age and gender. Through lightweight CNN classifiers, PCA analysis, speaker-level cross-validation, and cross-corpus evaluation, the work reveals for the first time that paralinguistic information is predominantly concentrated in the early to middle layers of these models. Experimental results demonstrate that HuBERT achieves the best performance in age recognition, while Wav2Vec2 and HuBERT lead in gender classification on the PFSTAR and CMU Kids datasets, respectively. Notably, robust and stable age and gender identification can be achieved using only 1–3 seconds of speech.

Abstract

Self-supervised learning (SSL) models have become a central component of modern speech processing systems, as they enable the learning of rich acoustic representations without reliance on labeled data. Despite their success on adult speech, it remains unclear how effectively these models capture speaker-related attributes such as age and gender in children's speech, which differs substantially from adult speech due to ongoing physiological and cognitive development. Higher pitch, increased articulatory variability, and age-dependent acoustic changes make children's speech a particularly challenging domain. In this work, we present a comprehensive analysis of how age and gender information is encoded across layers of four widely used SSL models: Wav2Vec2, HuBERT, Data2Vec, and WavLM. Layer-wise features are extracted and evaluated using a lightweight CNN on two benchmark children's speech corpora, PFSTAR and CMU Kids. To analyze feature compactness and redundancy, PCA is applied to identify redundancy and highlight the dimensions that contribute most to classification performance. Experimental results show that age- and gender-related information is unevenly distributed across SSL layers, with early to mid-level layers encoding the strongest paralinguistic cues. HuBERT achieves the best overall performance for age classification, while Wav2Vec2 and HuBERT lead gender classification on PFSTAR and CMU Kids, respectively. Beyond single-split evaluation, we further demonstrate that these findings remain stable under speaker-wise cross-validation, layer aggregation, and cross-database evaluation, indicating robustness to data imbalance and domain mismatch. Finally, we show that reliable age and gender classification is achievable even from short speech segments of 1--3 seconds.

Problem

• self-supervised speech models
• children's speech
• age
• gender
• paralinguistic attributes

Innovation

• self-supervised speech representation
• children's speech
• layer-wise analysis
• paralinguistic attribute encoding
• robust speaker profiling