Problem
Research questions and friction points this paper is trying to address.
Partially Relevant Video Retrieval
global contextual perception
query ambiguity
local noise
cross-modal matching
Imagine Before Concentration: Diffusion-Guided Registers Enhance Partially Relevant Video Retrieval
🤖 AI Summary
This work addresses the challenge of insufficient global contextual awareness in partial video retrieval, often caused by ambiguous queries and local noise. To tackle this issue, the authors propose a coarse-to-fine representation learning framework. Initially, a text-supervised truncated diffusion model generates semantic registers spanning the entire video to serve as coarse-grained highlights. Subsequently, a register-enhanced Gaussian attention mechanism refines cross-modal alignment at a fine-grained level. By integrating probabilistic variational sampling with diffusion-guided semantic registers, the method achieves substantial performance gains over state-of-the-art approaches across multiple benchmarks. The implementation code has been made publicly available.
📝 Abstract
Partially Relevant Video Retrieval (PRVR) aims to retrieve untrimmed videos based on text queries that describe only partial events. Existing methods suffer from incomplete global contextual perception, struggling with query ambiguity and local noise induced by spurious responses. To address these issues, we propose DreamPRVR, which adopts a coarse-to-fine representation learning paradigm. The model first generates global contextual semantic registers as coarse-grained highlights spanning the entire video and then concentrates on fine-grained similarity optimization for precise cross-modal matching. Concretely, these registers are generated by initializing from the video-centric distribution produced by a probabilistic variational sampler and then iteratively refined via a text-supervised truncated diffusion model. During this process, textual semantic structure learning constructs a well-formed textual latent space, enhancing the reliability of global perception. The registers are then adaptively fused with video tokens through register-augmented Gaussian attention blocks, enabling context-aware feature learning. Extensive experiments show that DreamPRVR outperforms state-of-the-art methods. Code is released at https://github.com/lijun2005/CVPR26-DreamPRVR.
Innovation
Methods, ideas, or system contributions that make the work stand out.
diffusion-guided registers
partially relevant video retrieval
coarse-to-fine representation
register-augmented attention
text-supervised diffusion
🔎 Similar Papers
No similar papers found.
