Thinking with Frames: Generative Video Distortion Evaluation via Frame Reward Model

2026-01-07

arXiv.org

AI Summary

Existing video reward models struggle to evaluate structural distortions in generated videos—such as anomalous object appearances and interactions—leading to incomplete quality assessments. To address this limitation, this work proposes REACT, the first frame-level interpretable reward model specifically designed for structural distortions. REACT employs dynamic frame sampling to focus on potential distortion regions and adopts a two-stage training strategy: first, supervised fine-tuning with a masked loss to inject domain knowledge, followed by GRPO-based reinforcement learning to align with human preferences. The project introduces a large-scale human preference dataset and chain-of-thought (CoT) synthetic data, augmented with attribution labels for precise distortion localization. Experiments on REACT-Bench demonstrate that REACT significantly improves both evaluation accuracy and interpretability, effectively complementing existing reward models.

Technology Category

• Computer Vision: Video Understanding & Activity Analysis
• Humans and AI: Human-Aware Planning and Behavior Prediction
• Machine Learning: Reinforcement Learning

Application Category

• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

Recent advances in video reward models and post-training strategies have improved text-to-video (T2V) generation. While these models typically assess visual quality, motion quality, and text alignment, they often overlook key structural distortions, such as abnormal object appearances and interactions, which can degrade the overall quality of the generative video. To address this gap, we introduce REACT, a frame-level reward model designed specifically for structural distortions evaluation in generative videos. REACT assigns point-wise scores and attribution labels by reasoning over video frames, focusing on recognizing distortions. To support this, we construct a large-scale human preference dataset, annotated based on our proposed taxonomy of structural distortions, and generate additional data using a efficient Chain-of-Thought (CoT) synthesis pipeline. REACT is trained with a two-stage framework: ((1) supervised fine-tuning with masked loss for domain knowledge injection, followed by (2) reinforcement learning with Group Relative Policy Optimization (GRPO) and pairwise rewards to enhance reasoning capability and align output scores with human preferences. During inference, a dynamic sampling mechanism is introduced to focus on frames most likely to exhibit distortion. We also present REACT-Bench, a benchmark for generative video distortion evaluation. Experimental results demonstrate that REACT complements existing reward models in assessing structutal distortion, achieving both accurate quantitative evaluations and interpretable attribution analysis.

Problem

• structural distortions
• generative video
• video evaluation
• object appearance
• abnormal interactions

Innovation

• frame-level reward model
• structural distortion evaluation
• Chain-of-Thought synthesis
• Group Relative Policy Optimization
• generative video benchmark