F2LLM-v2: Inclusive, Performant, and Efficient Embeddings for a Multilingual World

2026-03-19

AI Summary

This work addresses the limited support for low- and medium-resource languages and poor computational efficiency in existing large language models for multilingual embedding. We propose the F2LLM family of universal multilingual embedding models, covering over 200 languages, which uniquely integrates two-stage LLM embedding training, Matryoshka representation learning, model pruning, and knowledge distillation to substantially enhance inclusivity, performance, and efficiency. The F2LLM-v2-14B variant achieves state-of-the-art results across 11 MTEB benchmark tasks, while its lightweight counterparts establish new performance records under resource-constrained settings. All models and associated data are publicly released to foster further research and applications in multilingual representation learning.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: (Large) Language Models
• Knowledge Representation and Reasoning: Knowledge Representation Languages

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search

Abstract

We present F2LLM-v2, a new family of general-purpose, multilingual embedding models in 8 distinct sizes ranging from 80M to 14B. Trained on a newly curated composite of 60 million publicly available high-quality data samples, F2LLM-v2 supports more than 200 languages, with a particular emphasis on previously underserved mid- and low-resource languages. By integrating a two-stage LLM-based embedding training pipeline with matryoshka learning, model pruning, and knowledge distillation techniques, we present models that are far more efficient than previous LLM-based embedding models while retaining competitive performances. Extensive evaluations confirm that F2LLM-v2-14B ranks first on 11 MTEB benchmarks, while the smaller models in the family also set a new state of the art for resource-constrained applications. To facilitate open-source embedding model research, we release all models, data, code, and intermediate checkpoints.

Problem

• multilingual embeddings
• low-resource languages
• embedding efficiency
• language inclusivity
• large language models

Innovation

• multilingual embeddings
• matryoshka learning
• model pruning
• knowledge distillation
• low-resource languages