Leveraging LLM Tutoring Systems for Non-Native English Speakers in Introductory CS Courses

2024-11-05

arXiv.org

AI Summary

Non-native English-speaking (NNES) undergraduate students face significant linguistic and terminological barriers in introductory computer science courses. Method: We designed and empirically evaluated a pedagogically constrained, dialogue-based LLM teaching assistant specifically for NNES learners, incorporating multilingual code-switching comprehension, contextual terminology explanation, scaffolded problem-solving guidance, and anti-solution-dumping safeguards. Contribution/Results: First empirical evidence reveals that NNES students strongly prefer code-switched queries—embedding English programming keywords within Chinese sentences. Enrollment rates were statistically indistinguishable from native English speakers; 87% affirmed the system’s tolerance for non-fluent English and its efficacy in bridging gaps in domain-specific computational terminology absent in their L1. Students consistently preferred this assistant over conventional online resources for academic help. The study demonstrates that pedagogically grounded LLM tutors can differentially lower linguistic barriers, increase help-seeking propensity, and enhance equitable learning access for NNES learners.

Technology Category

• Natural Language Processing: Code Generation / Program Synthesis from Natural Language
• Machine Learning: Large Multimodal Models (LMMs)
• Humans and AI: AI for Accessibility

Application Category

• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

Computer science has historically presented barriers for non-native English speaking (NNES) students, often due to language and terminology challenges. With the rise of large language models (LLMs), there is potential to leverage this technology to support NNES students more effectively. Recent implementations of LLMs as tutors in classrooms have shown promising results. In this study, we deployed an LLM tutor in an accelerated introductory computing course to evaluate its effectiveness specifically for NNES students. Key insights for LLM tutor use are as follows: NNES students signed up for the LLM tutor at a similar rate to native English speakers (NES); NNES students used the system at a lower rate than NES students -- to a small effect; NNES students asked significantly more questions in languages other than English compared to NES students, with many of the questions being multilingual by incorporating English programming keywords. Results for views of the LLM tutor are as follows: both NNES and NES students appreciated the LLM tutor for its accessibility, conversational style, and the guardrails put in place to guide users to answers rather than directly providing solutions; NNES students highlighted its approachability as they did not need to communicate in perfect English; NNES students rated help-seeking preferences of online resources higher than NES students; Many NNES students were unfamiliar with computing terminology in their native languages. These results suggest that LLM tutors can be a valuable resource for NNES students in computing, providing tailored support that enhances their learning experience and overcomes language barriers.

Problem

• Evaluating LLM tutor effectiveness for non-native English speakers
• Addressing language barriers in introductory computing courses
• Analyzing usage patterns and preferences of NNES students

Innovation

• Deployed LLM tutor for non-native speakers
• Used multilingual question handling with programming keywords
• Implemented guided answers instead of direct solutions