RANKVIDEO: Reasoning Reranking for Text-to-Video Retrieval

2026-02-02

AI Summary

This work addresses the limitation of existing video retrieval systems, which lack reasoning-based reranking mechanisms and thus struggle with fine-grained relevance assessment. It introduces explicit reasoning into video reranking for the first time, proposing a perception-driven two-stage training framework. In the first stage, supervised fine-tuning with perception-aligned objectives enhances semantic understanding; in the second stage, a multi-objective optimization combines pointwise and pairwise losses, augmented by teacher-confidence distillation and synthetically generated hard reasoning samples. Evaluated on the MultiVENT 2.0 benchmark, the proposed method achieves an average 31% improvement in nDCG@10 over current text-only or vision-language reranking approaches, while maintaining higher computational efficiency.

Technology Category

• Computer Vision: Image and Video Retrieval
• Knowledge Representation and Reasoning: Argumentation
• Natural Language Processing: Sentence-level Semantics, Textual Inference, etc.

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Reranking is a critical component of modern retrieval systems, which typically pair an efficient first-stage retriever with a more expressive model to refine results. While large reasoning models have driven rapid progress in text-centric reranking, reasoning-based reranking for video retrieval remains underexplored. To address this gap, we introduce RANKVIDEO, a reasoning-based reranker for video retrieval that explicitly reasons over query-video pairs using video content to assess relevance. RANKVIDEO is trained using a two-stage curriculum consisting of perception-grounded supervised fine-tuning followed by reranking training that combines pointwise, pairwise, and teacher confidence distillation objectives, and is supported by a data synthesis pipeline for constructing reasoning-intensive query-video pairs. Experiments on the large-scale MultiVENT 2.0 benchmark demonstrate that RANKVIDEO consistently improves retrieval performance within a two-stage framework, yielding an average improvement of 31% on nDCG@10 and outperforming text-only and vision-language reranking alternatives, while more efficient.

Problem

• video retrieval
• reranking
• reasoning
• relevance assessment
• query-video pairs

Innovation

• reasoning-based reranking
• video retrieval
• curriculum training
• data synthesis
• vision-language reasoning