Vikhr: Constructing a State-of-the-art Bilingual Open-Source Instruction-Following Large Language Model for Russian
To address the poor generation quality and low computational efficiency of existing large language models for Russian, this paper introduces Vikhr—the first high-performance, bilingual, open-source instruction-following model natively optimized for Russian. Methodologically, Vikhr employs full-parameter continual pretraining followed by supervised instruction fine-tuning, deliberately avoiding parameter-efficient adaptations such as LoRA to achieve vocabulary-level native Russian support. Built upon the Mistral architecture, it features a custom Russian–English tokenizer, alongside substantial expansion of high-quality Russian instruction data and pretraining corpora, enhanced by multi-stage data cleaning and synthetic data generation. Experiments demonstrate that Vikhr establishes new state-of-the-art results among open-source models on multiple Russian-language benchmarks, with several metrics surpassing those of proprietary commercial models. All model weights, datasets, and training code are publicly released.