Latent Visual Cache for Video Reasoning

2026-07-01

AI Summary

This work addresses the visual anchoring degradation inherent in the “read-once, generate-many” paradigm of large-model video reasoning by introducing a recurrent latent visual caching mechanism embedded within the decoder. This approach maintains compact visual memory throughout the entire generation process. It integrates supervised contrastive cache alignment with a GRPO training strategy guided by latent visual rewards to ensure strict consistency between training and inference. Built upon the Qwen3.5-9B base model, the method leverages its native decoder hidden states to construct the visual cache. Evaluated across six video benchmarks, the proposed approach significantly outperforms strong baselines, particularly excelling in tasks that demand robust visual grounding and involve long-form videos, while achieving higher accuracy with shorter responses.

Abstract

Video reasoning requires Large Multimodal Models (LMMs) to remain grounded in dense evidence, yet existing systems largely adopt "read-once, generate-many" paradigm, in which visual grounding weakens during generation. This phenomenon has been widely observed and is known as Visual Anchoring Decay. To fill this gap, we introduce Latent Video Cache (Latent-VC), a recurrent latent visual cache inserted into the decoder to preserve compact visual memories throughout reasoning. The cache is trained with supervised contrastive cache alignment and vision-grounded GRPO with a latent grounding reward, while maintaining strict train-inference alignment through native decoder hidden states. Built on Qwen3.5-9B, Latent-VC consistently outperforms strong CoT and SFT+GRPO baselines across six video benchmarks, with especially clear gains on grounding-intensive and long-video tasks. In addition, it also achieves higher accuracy with substantially shorter responses, suggesting that latent visual caching improves video reasoning by preserving visual evidence rather than relying on longer textual chains.

Problem

• Video Reasoning
• Visual Anchoring Decay
• Latent Visual Cache
• Multimodal Models
• Visual Grounding

Innovation

• Latent Visual Cache
• Visual Anchoring Decay
• Video Reasoning
• Multimodal Models
• GRPO