Unhackable Temporal Rewarding for Scalable Video MLLMs

2025-02-17

AI Summary

Video multimodal large language models (MLLMs) exhibit a “reverse scaling law,” where increased model size and data volume degrade performance—rooted in “temporal hijacking”: overreliance on salient keyframes while neglecting temporal coherence. Method: We formally characterize this phenomenon from a reinforcement learning perspective, introducing Temporal Perplexity (TPL) as an interpretable metric for temporal modeling quality, and propose the Unhijackable Temporal Reward (UTR) framework to align agent objectives with genuine temporal understanding. Our approach seamlessly integrates temporal attention analysis, TPL-based quantification, and UTR-guided gradient correction into standard training pipelines. Contribution/Results: The method effectively suppresses temporal hijacking, yielding consistent performance gains of 12.6–23.4% across multiple video understanding benchmarks. TPL achieves a 0.89 correlation with human temporal annotations, validating its reliability as a principled indicator of temporal modeling fidelity.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: (Large) Language Models
• Computer Vision: Video Understanding & Activity Analysis

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

In the pursuit of superior video-processing MLLMs, we have encountered a perplexing paradox: the"anti-scaling law", where more data and larger models lead to worse performance. This study unmasks the culprit:"temporal hacking", a phenomenon where models shortcut by fixating on select frames, missing the full video narrative. In this work, we systematically establish a comprehensive theory of temporal hacking, defining it from a reinforcement learning perspective, introducing the Temporal Perplexity (TPL) score to assess this misalignment, and proposing the Unhackable Temporal Rewarding (UTR) framework to mitigate the temporal hacking. Both theoretically and empirically, TPL proves to be a reliable indicator of temporal modeling quality, correlating strongly with frame activation patterns. Extensive experiments reveal that UTR not only counters temporal hacking but significantly elevates video comprehension capabilities. This work not only advances video-AI systems but also illuminates the critical importance of aligning proxy rewards with true objectives in MLLM development.

Problem

• Addresses anti-scaling law in video MLLMs
• Identifies and mitigates temporal hacking
• Proposes Unhackable Temporal Rewarding framework

Innovation

• Unhackable Temporal Rewarding framework
• Temporal Perplexity score measurement
• Reinforcement learning-based temporal alignment