🤖 AI Summary
Existing bilingual vision-language models (VLMs) exhibit limitations in multi-image understanding and layout-aware OCR—i.e., joint modeling of textual content and its spatial coordinates. To address this, we propose VARCO-VISION-2.0, the first open-source Korean–English bilingual VLM. Methodologically, it introduces a novel four-stage curriculum learning strategy coupled with a memory-efficient training framework, integrating multimodal alignment enhancement, language capability preservation, safety optimization, and spatially aware OCR. Key contributions include: (1) robust bilingual understanding of complex multi-image inputs—including documents, charts, and tables; (2) release of two model variants—14B and 1.7B parameters—balancing high performance with edge-device deployability; and (3) strong empirical performance: the 14B model ranks eighth among same-scale models on the OpenCompass VLM leaderboard, with significant gains in bilingual spatial grounding accuracy.
📝 Abstract
We introduce VARCO-VISION-2.0, an open-weight bilingual vision-language model (VLM) for Korean and English with improved capabilities compared to the previous model VARCO-VISION-14B. The model supports multi-image understanding for complex inputs such as documents, charts, and tables, and delivers layoutaware OCR by predicting both textual content and its spatial location. Trained with a four-stage curriculum with memory-efficient techniques, the model achieves enhanced multimodal alignment, while preserving core language abilities and improving safety via preference optimization. Extensive benchmark evaluations demonstrate strong spatial grounding and competitive results for both languages, with the 14B model achieving 8th place on the OpenCompass VLM leaderboard among models of comparable scale. Alongside the 14B-scale model, we release a 1.7B version optimized for on-device deployment. We believe these models advance the development of bilingual VLMs and their practical applications. Two variants of VARCO-VISION-2.0 are available at Hugging Face: a full-scale 14B model and a lightweight 1.7B model.