DiCE-CIR: Direct Composition Learning for Efficient Zero-Shot Composed Image Retrieval

2026-07-06

AI Summary

This work addresses the challenges of training complexity, inefficiency, and train-inference inconsistency in zero-shot compositional image retrieval caused by projection and re-encoding steps. To overcome these limitations, the authors propose an end-to-end direct composition learning approach that eliminates conventional projection mechanisms. Instead, a lightweight composition module directly fuses reference images and editing text into a unified query representation. Training samples are automatically constructed from image–text pairs using a large language model, obviating the need for manually annotated triplets. The method integrates multi-objective contrastive learning—enforcing alignment, edit consistency, and retrieval discriminability—within a unified training and inference pipeline, thereby enhancing both efficiency and consistency. Experiments demonstrate state-of-the-art performance on CIRCO, competitive results on CIRR, and superior computational efficiency.

Abstract

Zero-shot composed image retrieval (ZS-CIR) aims to retrieve a target image from a multimodal query consisting of a reference image and an edit text describing the desired modification. Recent ZS-CIR studies have relied on projection-based methods that map a reference image into pseudo-word tokens in the text embedding space. However, such methods require additional projection and re-encoding steps, increasing training complexity, reducing efficiency, and introducing a discrepancy between training and inference. In this paper, we propose DiCE-CIR, a direct composition learning method that predicts composed query representations by directly composing a reference image and an edit text. To enable scalable training without manually annotated triplets, we automatically construct compositional training samples from large-scale image-caption pairs using a large language model. Based on these samples, we train a lightweight composition module with objectives that promote alignment with the target, edit-consistent semantic transformation, and retrieval discriminability. We conduct extensive experiments on ZS-CIR benchmarks and show that DiCE-CIR achieves state-of-the-art performance on CIRCO and competitive performance on CIRR while maintaining high computational efficiency.

Problem

• zero-shot composed image retrieval
• multimodal query
• reference image
• edit text
• training-inference discrepancy

Innovation

• Zero-shot Composed Image Retrieval
• Direct Composition Learning
• Large Language Model
• Efficient Multimodal Retrieval
• Composition Module