Fewer Steps, Better Performance: Efficient Cross-Modal Clip Trimming for Video Moment Retrieval Using Language

2026-05-28

AI Summary

This work addresses the limitations of existing video moment retrieval methods on ultra-long videos, where fixed sparse sampling often discards query-relevant frames, leading to temporal boundary inaccuracies and cross-modal misalignment. To overcome these issues, we propose SpotVMR—an efficient, plug-and-play, query-aware clip cropping framework. Its key innovations include a language-conditioned clip search mechanism, low-dimensional semantic indexing features for precise region localization, and a knowledge distillation–based loss function that effectively resolves optimization challenges in jointly training the cropping module with the downstream retrieval model. Extensive experiments demonstrate that SpotVMR substantially improves computational efficiency across three challenging benchmarks while maintaining or even surpassing state-of-the-art retrieval performance.

Abstract

Given an untrimmed video and a sentence query, video moment retrieval using language (VMR) aims to locate a target query-relevant moment. Since the untrimmed video is overlong, almost all existing VMR methods first sparsely down-sample each untrimmed video into multiple fixed-length video clips and then conduct multi-modal interactions with the query feature and expensive clip features for reasoning, which is infeasible for long real-world videos that span hours. Since the video is downsampled into fixed-length clips, some query-related frames may be filtered out, which will blur the specific boundary of the target moment, take the adjacent irrelevant frames as new boundaries, easily leading to cross-modal misalignment and introducing both boundary-bias and reasoning-bias. To this end, in this paper, we propose an efficient approach, SpotVMR, to trim the query-relevant clip. Besides, our proposed SpotVMR can serve as plug-and-play module, which achieves efficiency for state-of-the-art VMR methods while maintaining good retrieval performance. Especially, we first design a novel clip search model that learns to identify promising video regions to search conditioned on the language query. Then, we introduce a set of low-cost semantic indexing features to capture the context of objects and interactions that suggest where to search the query-relevant moment. Also, the distillation loss is utilized to address the optimization issues arising from end-to-end joint training of the clip selector and VMR model. Extensive experiments on three challenging datasets demonstrate its effectiveness.

Problem

• video moment retrieval
• cross-modal alignment
• clip trimming
• boundary bias
• long untrimmed videos

Innovation

• video moment retrieval
• cross-modal alignment
• efficient clip trimming
• semantic indexing
• distillation loss