The First ChineseBabyLM Challenge: training data-efficient and cognitively plausible language models for Chinese

2026-07-12

AI Summary

This work proposes the Chinese BabyLM Challenge, which aims to train language models from scratch using only 100 million Chinese tokens—a highly constrained data scale—to achieve strong capabilities in natural language understanding, cognitive plausibility, and Chinese character knowledge. Participants are free to design tokenizers, model architectures, and training strategies, with an emphasis on data-efficient learning and human-like cognitive modeling. The challenge introduces a comprehensive evaluation framework comprising three tracks: natural language understanding, cognitive alignment, and character-level knowledge. This is the first benchmark specifically tailored for Chinese that prioritizes data efficiency and cognitive fidelity in lightweight language models, challenging the prevailing paradigm of relying on massive datasets for large language models and advancing the development of efficient, interpretable Chinese foundation models.

Abstract

This paper describes the first ChineseBabyLM challenge, which will be held in the 2026 NLPCC conference. The challenge calls for researchers to train language models from scratch with 100 million Chinese tokens and evaluates the models on 3 tracks of tasks: NLU, cognitive alignment and Hanzi knowledge. There is no restriction on tokenizer, model architecture and the number of training epochs. Details of the challenge can be found in https://chinese-babylm.github.io/.

Problem

• data-efficient
• cognitively plausible
• language models
• Chinese
• limited training data

Innovation

• data-efficient
• cognitively plausible
• Chinese language modeling
• BabyLM challenge
• from-scratch training