STRIDE: Strategic Trajectory Reasoning via Discriminative Estimation for Verifiable Reinforcement Learning

2026-06-14

AI Summary

This work addresses the limitations of existing verifiable reinforcement learning (RLVR), which relies on sparse supervision from final answers and struggles to assign fine-grained credit to individual tokens during reasoning due to unverifiable intermediate signals. The authors propose STRIDE, a novel framework that transforms verifiable outcomes into fine-grained policy supervision by contrasting successful and failed trajectories to estimate discriminative preferences over n-gram policy patterns. STRIDE further identifies critical decision points through reasoning saliency entropy, enabling precise credit assignment while preserving verifiability. Experimental results demonstrate that STRIDE significantly enhances reasoning performance across diverse models, tasks, and extended settings—including vision-language models and agent systems—thereby validating its effectiveness and strong generalization capability.

Abstract

Reinforcement Learning with Verifiable Rewards (RLVR) has become an effective post-training paradigm for improving the reasoning abilities of large language models. However, existing RLVR methods typically rely on final-answer correctness to assign trajectory-level rewards, providing sparse supervision and treating all tokens uniformly regardless of their actual contribution to reasoning. Although recent studies introduce intermediate signals such as process rewards, high-entropy tokens, and semantic uncertainty, these signals are often not inherently verifiable and may fail to distinguish beneficial strategic patterns from harmful ones. To address this limitation, we propose STRIDE (Strategic Trajectory Reasoning with Discriminative Estimation), a fine-grained RLVR framework that derives strategic reasoning supervision from verifiable outcomes. STRIDE contrasts successful and failed trajectories within each response group to estimate the outcome-discriminative preference of each $n$-gram strategic pattern, and further combines this signal with reasoning saliency entropy to identify decision-relevant strategic patterns. These patterns are assigned differentiated advantage values during RL optimization, enabling more precise credit assignment while preserving the verifiability of RLVR. Extensive experiments demonstrate that STRIDE consistently improves reasoning performance across diverse models, tasks, and extended settings, including VLMs and agent-based systems.

Problem

• Reinforcement Learning with Verifiable Rewards
• trajectory-level rewards
• strategic reasoning
• credit assignment
• verifiability

Innovation

• STRIDE
• Verifiable Reinforcement Learning
• Strategic Trajectory Reasoning
• Discriminative Estimation
• Credit Assignment