Prompting ChatGPT for Chinese Learning as L2: A CEFR and EBCL Level Study

2025-01-25

AI Summary

This study investigates the efficacy of large language models (LLMs) in supporting beginner-level Chinese as a second language (CSL) learners, specifically those at CEFR A1–A2 and corresponding EBCL proficiency levels, with respect to oral production and Chinese character acquisition. Method: We propose a competence-framework–driven, tiered prompt engineering approach: for the first time, we precisely align the EBCL Chinese character benchmark list with CEFR levels within ChatGPT prompts, designing interactive exercises integrating listening, speaking, reading, and writing—while controlling lexical repetition frequency and incorporating character-writing constraints. Language generation is hierarchically constrained via high-frequency character lexicons, spoken-vocabulary output limits, and verifiable response formats. Contribution/Results: Experimental results show that A1/A1+-level character-set prompts significantly improve model adherence to the EBCL character list, increase target-input density, and enhance interaction effectiveness—demonstrating the feasibility and pedagogical value of controllable, character-informed LLM generation for CSL instruction.

Technology Category

• Natural Language Processing: (Large) Language Models
• Machine Learning: Large Multimodal Models (LMMs)
• Planning, Routing, and Scheduling: Planning with Language Models

Application Category

• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information

Abstract

The use of chatbots in language learning has evolved significantly since the 1960s, becoming more sophisticated platforms as generative AI emerged. These tools now simulate natural conversations, adapting to individual learners' needs, including those studying Chinese. Our study explores how learners can use specific prompts to engage Large Language Models (LLM) as personalized chatbots, aiming to target their language level based on the Common European Framework of Reference for Languages (CEFR) and the European Benchmarking Chinese Language (EBCL) project. Focusing on A1, A1+ and A2 levels, we examine the teaching of Chinese, which presents unique challenges due to its logographic writing system. Our goal is to develop prompts that integrate oral and written skills, using high-frequency character lists and controlling oral lexical productions. These tools, powered by generative AI, aim to enhance language practice by crossing lexical and sinographic recurrence. While generative AI shows potential as a personalized tutor, further evaluation is needed to assess its effectiveness. We conducted a systematic series of experiments using ChatGPT models to evaluate their adherence to constraints specified in the prompts. The results indicate that incorporating level A1 and A1+ characters, along with the associated reference list, significantly enhances compliance with the EBCL character set. Properly prompted, LLMs can increase exposure to the target language and offer interactive exchanges to develop language skills.

Problem

• Language Models
• Chinese Second Language Learning
• Oral and Character Acquisition

Innovation

• ChatGPT
• Chinese Language Learning
• Personalized Learning