Beyond Film Subtitles: Is YouTube the Best Approximation of Spoken Vocabulary?

2024-10-04

arXiv.org

AI Summary

Low-resource languages lack high-quality spoken-word frequency data for psycholinguistic research. Method: This study systematically validates YouTube subtitles as a scalable, low-cost source of psycholinguistic word frequency norms. We curate and clean multilingual (Chinese, English, Indonesian, Japanese, Spanish) YouTube subtitle corpora, construct the first large-scale spoken-word frequency database, and train fastText embeddings to support cross-lingual modeling. Contribution/Results: Empirical evaluation shows strong correlations between our frequencies and lexical decision reaction times and subjective familiarity ratings. In English and Japanese lexical complexity prediction, our model outperforms both GPT-4 and traditional film subtitle baselines, achieving new state-of-the-art performance. Crucially, this is the first work to demonstrate that YouTube subtitles yield superior psycholinguistic validity compared to film subtitles. All code, frequency tables, and pre-trained models are publicly released, establishing an extensible, cost-effective data infrastructure for cognitive modeling in low-resource languages.

Technology Category

• Natural Language Processing: Lexical Semantics and Morphology
• Machine Learning: Large Multimodal Models (LMMs)
• Planning, Routing, and Scheduling: Planning with Language Models

Application Category

• Web Mining and Content Analysis: Mining multimedia, multimodal, multilingual, cross-lingual Web data
• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets

Abstract

Word frequency is a key variable in psycholinguistics, useful for modeling human familiarity with words even in the era of large language models (LLMs). Frequency in film subtitles has proved to be a particularly good approximation of everyday language exposure. For many languages, however, film subtitles are not easily available, or are overwhelmingly translated from English. We demonstrate that frequencies extracted from carefully processed YouTube subtitles provide an approximation comparable to, and often better than, the best currently available resources. Moreover, they are available for languages for which a high-quality subtitle or speech corpus does not exist. We use YouTube subtitles to construct frequency norms for five diverse languages, Chinese, English, Indonesian, Japanese, and Spanish, and evaluate their correlation with lexical decision time, word familiarity, and lexical complexity. In addition to being strongly correlated with two psycholinguistic variables, a simple linear regression on the new frequencies achieves a new high score on a lexical complexity prediction task in English and Japanese, surpassing both models trained on film subtitle frequencies and the LLM GPT-4. Our code, the frequency lists, fastText word embeddings, and statistical language models are freely available at https://github.com/naist-nlp/tubelex.

Problem

• YouTube subtitles
• lexical frequency
• language proficiency prediction

Innovation

• YouTube subtitles
• Vocabulary frequency prediction
• Psycholinguistic research