SimClass: A Classroom Speech Dataset Generated via Game Engine Simulation For Automatic Speech Recognition Research

2025-06-10

AI Summary

A critical bottleneck in educational ASR is the scarcity of large-scale, high-fidelity classroom speech data; existing public datasets are small and lack realistic noise conditions, limiting model robustness in authentic teaching environments. To address this, we propose a scalable acoustic modeling framework for classroom scenes built upon game-engine-based physical audio simulation. We introduce SimClass—the first open-source synthetic classroom speech benchmark—featuring a diverse classroom noise library and child-teacher interactive speech samples. Our approach innovatively integrates physics-based acoustic simulation, speech-video temporal alignment and synthesis, child voice conversion, and YouTube educational video–driven contextual speech generation. Experiments demonstrate that SimClass closely approximates real classroom speech distributions under both clean and noisy conditions, and consistently improves ASR accuracy and noise robustness across multiple models in educational settings.

Technology Category

• Natural Language Processing: Speech
• Cognitive Modeling & Cognitive Systems: Simulating Human Behavior
• Machine Learning: Large Multimodal Models (LMMs)

Application Category

• User Modeling, Personalization and Recommendation: User modeling and simulation for interactive and conversational systems
• Search and Retrieval-Augmented AI: Large language models for search
• Web Mining and Content Analysis: Web data generation and simulation

Abstract

The scarcity of large-scale classroom speech data has hindered the development of AI-driven speech models for education. Public classroom datasets remain limited, and the lack of a dedicated classroom noise corpus prevents the use of standard data augmentation techniques. In this paper, we introduce a scalable methodology for synthesizing classroom noise using game engines, a framework that extends to other domains. Using this methodology, we present SimClass, a dataset that includes both a synthesized classroom noise corpus and a simulated classroom speech dataset. The speech data is generated by pairing a public children's speech corpus with YouTube lecture videos to approximate real classroom interactions in clean conditions. Our experiments on clean and noisy speech demonstrate that SimClass closely approximates real classroom speech, making it a valuable resource for developing robust speech recognition and enhancement models.

Problem

• Lack of large-scale classroom speech data for AI models
• Absence of dedicated classroom noise corpus for augmentation
• Need for realistic simulated classroom speech datasets

Innovation

• Synthesize classroom noise using game engines
• Pair children's speech with lecture videos
• SimClass dataset mimics real classroom speech