MoVA: Learning Asymmetric Dual Projections for Modular Long Video-Text Alignment

2026-07-01

AI Summary

This work addresses the challenge of representation entanglement arising from temporal misalignment and semantic asymmetry in aligning long videos with detailed textual descriptions. To this end, the authors propose a modular alignment framework that establishes, for the first time, theoretical conditions enabling flexible video–text alignment. Central to this framework is a dual asymmetric projection mechanism: on the text side, it employs adaptive subspace selection, while on the video side, it decouples visual concepts to facilitate precise alignment across both temporal dimensions and multiple granularity levels. Built upon contrastive pretraining, the resulting dual asymmetric projection network significantly outperforms existing methods across various video–text tasks, achieving scalable and high-precision alignment between long-form videos and extended textual narratives.

Abstract

Contrastive pre-training has propelled video-text alignment, yet models often inherit the critical limitations of their image-text predecessors like CLIP, resulting in entangled representations. These challenges are severely exacerbated by two fundamental properties in the video domain: Temporal Misalignment, where textual descriptions often correlate only to specific, constrained temporal windows, leaving other frames text-irrelevant; and Semantic Asymmetry, which dictates a sparse, bidirectional, and non-equivalent relevance between frame-level visual details and caption-level concepts. This failure persists whether captions are short and temporally disjoint, creating ambiguity, or long and detailed, fostering entanglement between static objects and their temporal evolution. In this paper, we establish theoretical conditions that enable flexible alignment between video and text representations across the temporal dimension and at varying levels of granularity. Building on these theoretical insights, we introduce MoVA, Modular Long Video-Text Alignment, which learns dual asymmetric projections: a text-side projection that adaptively selects frame-aware subspaces of the caption, and a video-side projection that disentangles text-relevant visual concepts. Our framework ensures that the model can preserve global cross-modal semantics while disentangling evolving, frame-specific concepts and scale naturally to long captions and videos. Empirical evaluations show that MoVA outperforms existing methods in multiple video-text alignment tasks, demonstrating the effectiveness of our method.

Problem

• Temporal Misalignment
• Semantic Asymmetry
• Video-Text Alignment
• Representation Entanglement

Innovation

• asymmetric dual projections
• temporal misalignment
• semantic asymmetry
• modular alignment
• video-text disentanglement