VideoLatent: Video-Language Learning via Latent Self-Forcing

2026-06-22

AI Summary

This work addresses the limitations of existing video-language models, which rely on costly human annotations or additional supervision, leading to high computational overhead and poor scalability. The authors propose VideoLatent, the first implicit self-distillation training paradigm tailored for video understanding, which leverages only standard video-question-answer triplets without any extra supervision to generate aligned yet diverse visual latent representations. By integrating an implicit injection module and a self-distillation strategy, VideoLatent is compatible with various multimodal large language model architectures. Evaluated across 14 benchmarks, the method consistently outperforms current approaches; compared to Video-R1, it reduces training and inference costs by approximately 6× and 68×, respectively, substantially enhancing efficiency, scalability, and generalization.

Abstract

Recent advancements in chain-of-thought (CoT) reasoning have shown promise in enhancing video understanding and reasoning capabilities of multimodal large language models (MLLMs). However, existing CoT-based MLLMs require labor-intensive CoT annotations and incur substantial training and inference overhead. While visual latent reasoning has emerged as a more efficient alternative, existing methods primarily focus on image tasks and heavily rely on additional supervision signals for visual latent generation (e.g., CoT traces, auxiliary images, or fine-grained annotations), limiting their scalability and transferability to video tasks. To bridge this gap, we introduce VideoLatent, a novel MLLM equipped with a latent injection module tailored for video understanding and reasoning. Specifically, VideoLatent learns to perform visual latent reasoning using a new latent self-forcing training paradigm, which comprises latent alignment and latent diversity objectives, and relies solely on standard video-question-answer triplets. Extensive experiments across 14 benchmarks demonstrate that our model consistently outperforms existing standard and latent MLLMs on general video understanding and complex video reasoning. Compared with Video-R1, our VideoLatent achieves superior computational efficiency, reducing training/inference overhead by $\sim$6$\times$/$\sim$68$\times$. Moreover, experiments demonstrate that our method has strong generalizability to different MLLM backbones and different model scales.

Problem

• video understanding
• multimodal large language models
• latent reasoning
• chain-of-thought
• supervision signals

Innovation

• latent self-forcing
• video-language learning
• visual latent reasoning
• multimodal large language models
• computational efficiency