VlogQA: Task, Dataset, and Baseline Models for Vietnamese Spoken-Based Machine Reading Comprehension

2024-02-05

Conference of the European Chapter of the Association for Computational Linguistics

AI Summary

Existing Vietnamese machine reading comprehension (MRC) datasets heavily rely on formal written texts (e.g., Wikipedia, news), lacking resources grounded in authentic spoken language. To address this gap, we introduce VlogQA—the first natural spoken-language Vietnamese MRC dataset—comprising 10,076 question-answer pairs derived from transcriptions of 1,230 YouTube videos covering food and travel genres. To handle spoken-language phenomena such as ambiguity, ellipsis, and prosodic variation, we propose a post-processing pipeline for speech transcripts and a context-enrichment strategy. We further employ multi-task fine-tuning using PhoBERT-based models. Experiments show that our approach achieves 75.34 F1 and 53.97 EM on the VlogQA test set, substantially advancing Vietnamese spoken-MRC performance. This work bridges dual gaps in low-resource spoken-MRC research: it provides the first benchmark dataset and an effective methodology tailored to spontaneous speech, establishing a new paradigm for real-world spoken-language understanding.

Technology Category

• Natural Language Processing: Speech
• Machine Learning: Large Multimodal Models (LMMs)
• Data Mining & Knowledge Management: Conversational Systems for Recommendation & Retrieval

Application Category

• Web Mining and Content Analysis: Large pretrained models with web data
• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

This paper presents the development process of a Vietnamese spoken language corpus for machine reading comprehension (MRC) tasks and provides insights into the challenges and opportunities associated with using real-world data for machine reading comprehension tasks. The existing MRC corpora in Vietnamese mainly focus on formal written documents such as Wikipedia articles, online newspapers, or textbooks. In contrast, the VlogQA consists of 10,076 question-answer pairs based on 1,230 transcript documents sourced from YouTube – an extensive source of user-uploaded content, covering the topics of food and travel. By capturing the spoken language of native Vietnamese speakers in natural settings, an obscure corner overlooked in Vietnamese research, the corpus provides a valuable resource for future research in reading comprehension tasks for the Vietnamese language. Regarding performance evaluation, our deep-learning models achieved the highest F1 score of 75.34% on the test set, indicating significant progress in machine reading comprehension for Vietnamese spoken language data. In terms of EM, the highest score we accomplished is 53.97%, which reflects the challenge in processing spoken-based content and highlights the need for further improvement.

Problem

• Develop Vietnamese spoken language corpus for machine reading comprehension
• Address lack of real-world spoken data in Vietnamese MRC research
• Evaluate deep learning models on spoken language comprehension challenges

Innovation

• Vietnamese spoken language corpus development
• Deep-learning models for reading comprehension
• YouTube-sourced transcripts for training data