Gemini Embedding 2: A Native Multimodal Embedding Model from Gemini

2026-05-26

AI Summary

This work proposes a unified embedding model natively supporting arbitrary interleaved combinations of video, audio, image, and text modalities to construct a general-purpose multimodal representation for cross-modal and multimodal tasks. Built upon the Gemini architecture, the approach leverages large-scale contrastive learning, multi-task multi-stage training, and a unified embedding space to achieve zero-shot cross-domain generalization without fine-tuning. The model surpasses specialized architectures across diverse benchmarks, including MSCOCO (R@1=62.9), Vatex (NDCG@10=68.8), MTEB multilingual (69.9), and code retrieval (84.0), significantly enhancing downstream performance in retrieval, recommendation, and retrieval-augmented generation (RAG) tasks.

Abstract

We introduce Gemini Embedding 2, a native multimodal embedding model that allows embedding video, audio, image, and text modalities in a unified representation space. We leverage the multimodal capabilities of Gemini to produce embeddings for arbitrary combinations of interleaved inputs across all these modalities that generalize well across a wide variety of tasks. Applying large-scale contrastive learning in a multi-task multi-stage training setup, we achieve state-of-the-art performance on key embedding benchmarks including unimodal, cross-modal, and multimodal retrieval spanning a diverse set of tasks. We show that our embedding model demonstrates strong performance (with a score of 62.9 R@1 on MSCOCO, 68.8 NDCG@10 on Vatex, 69.9 on MTEB multilingual and 84.0 on MTEB Code) across a variety of tasks surpassing the performance of specialized models. These unified capabilities make Gemini Embedding 2 a promising candidate for downstream use cases such as RAG, recommendation and search. Furthermore, its robust zero-shot performance across distinct fields - from astronomy and bioscience to fine arts and the culinary arts - establishes it as a highly reliable, out-of-the-box representation even for specialized domains.

Problem

• multimodal embedding
• unified representation
• cross-modal retrieval
• zero-shot generalization
• embedding model

Innovation

• multimodal embedding
• contrastive learning
• zero-shot generalization
• unified representation space
• multi-task training