Reasoning as Intersection: Consensus-Frame Alignment for Visual Focus in Video-MLLMs

2026-06-16

AI Summary

This work addresses the challenge that existing outcome-reward-based reinforcement learning methods struggle to guide video multimodal large language models (Video-MLLMs) toward focusing on critical visual evidence. The authors propose Consensus Frame GRPO (CF-GRPO), a novel framework that, for the first time, incorporates multisensory integration theory into video reasoning. CF-GRPO constructs a consensus frame prior without requiring manual temporal annotations and aligns it with the model’s frame utilization scores through a high-contrast reward mechanism. The consensus prior is derived from temporal coverage, scene transitions, and query relevance, while consensus frame rewards (CFR) are computed via sparse aggregation and distribution sharpening. This approach significantly enhances the model’s evidence awareness and interpretability. Experiments demonstrate that CF-GRPO outperforms state-of-the-art Video-MLLMs and reinforcement learning baselines across multiple complex video reasoning benchmarks, with visualizations confirming its effective focus on key evidence frames.

Abstract

Reinforcement learning has improved the reasoning ability of large language models, but applying outcome-only rewards to video multimodal large language models (Video-MLLMs) provides limited guidance on which visual evidence should support the answer. Inspired by multisensory integration, where consistent cues can enhance the salience and reliability of perceptual estimates, we introduce Consensus Frame GRPO (CF-GRPO), a temporal-annotation-free process-level reward framework for evidence-aware video reasoning. CF-GRPO constructs a consensus frame prior from intrinsic video cues, including temporal coverage, scene-transition cues, and query-conditioned visual relevance. It then computes a model-side frame-use score from visual and response representations and optimizes their agreement through the Consensus Frame Reward (CFR). With salience-aware sparse aggregation and distribution sharpening, CFR provides a high-contrast reward signal without requiring human temporal annotations. Experiments show that VideoCFR achieves competitive performance across complex video reasoning benchmarks and improves several metrics over representative Video-MLLM and RL baselines, while the consensus prior provides an interpretable view of the evidence frames emphasized during training. The implementation is available at https://github.com/1Pansy/VideoCFR.

Problem

• Video-MLLMs
• visual evidence
• reasoning
• reward signal
• temporal annotation

Innovation

• Consensus Frame Reward
• Video-MLLMs
• Evidence-aware Reasoning
• Temporal-annotation-free RL
• Multisensory Integration