Learning Sample-wise Rank-aware Interpolation Weights for Composed Visual Data Retrieval

2026-08-23

AI Summary

本文提出SRAIN框架，通过动态预测查询特定的插值权重来解决组合视觉数据检索中的查询时间延迟问题，使用批量排名感知权重估计和紧凑记忆库生成难负样本。

Abstract

At the heart of composed visual data retrieval is the fusion of a reference visual input and a textual modification into a single query. While current state-of-the-art methods utilize multimodal large language models for this fusion, their complexity introduces prohibitive querytime latency, limiting their scalability. We instead revisit the efficacy of simple linear interpolation within an embedding space, and introduce SRAIN, the first framework that dynamically predicts query-specific interpolation weights. The key challenge lies in the fact that the quality of an interpolation weight should be measured by the interpolated embedding's discriminability from negatives as well as its proximity to true targets; this makes collecting and predicting optimal weights intractable. We overcome this bottleneck through two key innovations: batch-wise rank-aware weight estimation during training, and a compact memory bank that synthesizes hard negatives during inference. SRAIN achieves the best in composed video retrieval and matches the current state of the art in composed image retrieval, all while substantially reducing querytime latency compared to MLLM-based alternatives.

Problem

• composed visual data retrieval
• query-time latency
• interpolation weights

Innovation

• SRAIN
• rank-aware weight estimation
• compact memory bank
• query-specific interpolation weights