Unsupervised Learning and Representation of Mandarin Tonal Categories by a Generative CNN

2025-09-22

AI Summary

This study addresses the challenge of modeling Mandarin tone categories from unlabeled speech data, aiming to emulate human tonal acquisition through unsupervised generative modeling. We propose a generative framework integrating conditional improved Wasserstein GAN (ciWGAN) with convolutional neural networks, augmented by F0 statistical analysis and latent-space visualization. To our knowledge, this is the first work to construct tone representations aligned with developmental stages of human language acquisition under fully unsupervised conditions. A novel internal representation tracking method—applied layer-wise within convolutional modules—enhances model interpretability. Experiments demonstrate that a model trained exclusively on male speech robustly disentangles the four tones in its latent space, effectively replicating human-like tonal contrast perception. Our work establishes a new paradigm for unsupervised speech representation learning and advances the cognitive interpretability of deep generative models.

Technology Category

• Natural Language Processing: Generation
• Machine Learning: Deep Generative Models & Autoencoders
• Computer Vision: Generative Adversarial Networks (GANs) for Vision

Application Category

• User Modeling, Personalization and Recommendation: User modeling and simulation for interactive and conversational systems
• Economics, Online Markets and Human Computation: Economic ramifications for generative AI infrastructure and applications
• Web Mining and Content Analysis: Web data generation and simulation

Abstract

This paper outlines the methodology for modeling tonal learning in fully unsupervised models of human language acquisition. Tonal patterns are among the computationally most complex learning objectives in language. We argue that a realistic generative model of human language (ciwGAN) can learn to associate its categorical variables with Mandarin Chinese tonal categories without any labeled data. All three trained models showed statistically significant differences in F0 across categorical variables. The model trained solely on male tokens consistently encoded tone. Our results sug- gest that not only does the model learn Mandarin tonal contrasts, but it learns a system that corresponds to a stage of acquisition in human language learners. We also outline methodology for tracing tonal representations in internal convolutional layers, which shows that linguistic tools can contribute to interpretability of deep learning and can ultimately be used in neural experiments.

Problem

• Modeling unsupervised tonal learning in language acquisition
• Learning Mandarin tonal categories without labeled training data
• Analyzing internal representations using interpretable linguistic tools

Innovation

• Generative CNN models tonal categories unsupervised
• Categorical variables associate with Mandarin tones
• Convolutional layer analysis enables linguistic interpretability