Dango: A Strictly L1-Only Large Language Model for Studying Second Language Acquisition

2026-06-17

AI Summary

This study addresses the challenge that existing large language models struggle to effectively simulate authentic second-language (L2) learners due to model scale, architectural constraints, and target-language contamination in pretraining corpora. To overcome this, the authors construct a 1.8B-parameter decoder-only model pretrained exclusively on Japanese (L1) data and introduce a novel L2 contamination filtering method tailored for large-scale models. By minimizing premature exposure to English, they achieve a pristine L1 pretraining regime and, for the first time, successfully emulate L1→L2 transfer within a pure decoder architecture. Through multi-stage corpus filtering and LLM-generated L2 curriculum fine-tuning, the model exhibits human-like English production patterns and significantly outperforms both unfiltered counterparts and standard multilingual baselines across multiple evaluations, establishing a reproducible paradigm for computational second-language acquisition research and educational applications.

Abstract

We introduce Dango, a 1.8B-parameter large language model designed for controlled studies of L1-to-L2 (Japanese-to-English) transfer in second language acquisition (SLA). While previous studies have explored SLA in language models, they have predominantly relied on smaller or non-decoder models, limiting their ability to generate open-ended text and reducing their suitability as practical L2 simulators. We identify a key challenge when scaling models to this size: L2 contamination within the "monolingual" pretraining corpus used for L1 acquisition. To address this, we propose a filtering method to reduce premature exposure to English while preserving realistic, minimal exposure. We then fine-tune the model on LLM-generated L2-learning lessons to simulate the L2 acquisition process. Our evaluations confirm that Dango develops human-like L2 production patterns, outperforming both unfiltered and standard multilingual baselines. We release the model, data, and code to facilitate reproducible computational SLA research and learner-facing applications.

Problem

• second language acquisition
• L1-to-L2 transfer
• L2 contamination
• monolingual pretraining
• language model

Innovation

• second language acquisition
• L1-only pretraining
• corpus filtering
• language transfer
• LLM-based simulation