Claim-Level Rubric Rewards for Video Caption Reinforcement Learning

2026-07-06

AI Summary

Existing reinforcement learning approaches for dense video captioning struggle with reward design, often failing to simultaneously ensure factual accuracy, generation diversity, and evaluation reliability. This work proposes CuRe, a structured reward framework that reframes reward modeling as a fine-grained verification task at the level of atomic statements. By decomposing captions into category-aware atomic assertions and applying structured scoring rules to validate each statement individually, CuRe replaces conventional holistic or alignment-based rewards. This approach effectively mitigates reward hacking while significantly improving the factual accuracy and completeness of generated descriptions, without compromising linguistic diversity.

Abstract

In this paper, we introduce Claim-Level Rubric Rewards (CuRe), a structured reward framework designed to address the reward-design bottleneck in reinforcement learning for dense video captioning. Existing reward designs generally fall into two categories: holistic response-level judgment across heterogeneous criteria, or alignment-based evaluation against reference captions. However, both paradigms suffer from fundamental limitations. Holistic rewards struggle to ensure factual accuracy and are prone to stylistic reward hacking, while reference-based rewards overly rely on rigid textual alignment, failing to preserve the completeness and diversity inherent to open-ended generation tasks. To address these challenges, CuRe reformulates reward modeling as fine-grained claim-level verification. Specifically, CuRe decomposes captions into category-aware atomic claims through a structured rubric, converting holistic evaluation into simpler and more reliable claim-level verification.

Problem

• dense video captioning
• reinforcement learning
• reward design
• factual accuracy
• textual alignment

Innovation

• Claim-Level Verification
• Structured Reward Framework
• Dense Video Captioning
• Reinforcement Learning
• Rubric-Based Evaluation