Problem
Research questions and friction points this paper is trying to address.
Tajik language
lightweight foundation model
low-resource deployment
language benchmark
conversational AI
Soro: A Lightweight Foundation Model and Chatbot for Tajik
🤖 AI Summary
This work addresses the scarcity of lightweight large language models for Tajik by leveraging Gemma-3 checkpoints to develop the first dedicated, lightweight conversational model for the language. The approach involves continued pretraining on a 1.9-billion-word multilingual Tajik corpus followed by supervised fine-tuning on 40,000 instruction-style examples curated in an educational format. Concurrently, the authors introduce and open-source a comprehensive Tajik evaluation benchmark spanning general knowledge and educational assessment domains. Through FP8/INT4 quantization, the resulting model maintains competitive performance while being deployable in resource-constrained settings. It significantly outperforms the baseline Gemma-3 model of comparable scale on the newly established benchmark and has already been integrated into pilot educational programs in Tajikistan.
📝 Abstract
We present Soro, a family of Tajik-specialized conversational large language models (LLMs) designed for real-world deployment under tight compute and connectivity constraints in Tajikistan. Starting from open-weight Gemma 3 checkpoints, we perform Tajik-only continual pretraining on a curated 1.9-billion-token corpus spanning filtered web text, PDF documents, and curriculum-aligned educational materials, followed by supervised instruction tuning on 40K Tajik teacher-style examples. To enable rigorous evaluation despite the limited coverage of Tajik in standard benchmarks, we introduce a suite of Tajik benchmarks covering general knowledge, linguistic competence, and school- and university entrance-exam domains, and we open-source them on Hugging Face. Across these Tajik benchmarks, Soro substantially outperforms same-size Gemma 3 baselines while retaining strong English performance on standard datasets. We further show that FP8 and INT4 quantization of Soro preserves most Tajik-language gains while reducing memory requirements for edge deployment, supporting an ongoing education-sector pilot and planned scale-out across schools in Tajikistan.
Innovation
Methods, ideas, or system contributions that make the work stand out.
Tajik-specialized LLM
continual pretraining
instruction tuning
low-resource language benchmark
model quantization
🔎 Similar Papers
No similar papers found.
💼 Related Jobs
