ReCast: Recasting Learning Signals for Reinforcement Learning in Generative Recommendation

2026-04-23

AI Summary

This work addresses the failure of group-based reinforcement learning in generative recommendation under sparse reward scenarios, where groups receiving all-zero rewards lack effective learning signals. The authors propose ReCast, a novel framework that decouples signal construction into a two-stage “repair-and-contrast” process: first restoring minimal learnability for all-zero groups, then applying margin-focused contrastive learning to strengthen the most confident positive and hardest negative examples within each group—thereby reconstructing intra-group signals without modifying the external RL architecture. This approach partially decouples rollout search width from policy update width, yielding substantial efficiency gains across multiple tasks: Pass@1 improves by up to 36.6%, baseline performance is matched using only 4.1% of the rollout budget, actor update time is reduced by 16.6×, peak memory consumption drops by 16.5%, and model FLOPs utilization (MFU) increases by 14.2%.

Technology Category

• Search and Optimization: Learning to Search
• Machine Learning: Reinforcement Learning
• Planning, Routing, and Scheduling: Replanning and Plan Repair

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Economics, Online Markets and Human Computation: Trust and reliance of crowd workers and data experts on GenAI

Abstract

Generic group-based RL assumes that sampled rollout groups are already usable learning signals. We show that this assumption breaks down in sparse-hit generative recommendation, where many sampled groups never become learnable at all. We propose ReCast, a repair-then-contrast learning-signal framework that first restores minimal learnability for all-zero groups and then replaces full-group reward normalization with a boundary-focused contrastive update on the strongest positive and the hardest negative. ReCast leaves the outer RL framework unchanged, modifies only within-group signal construction, and partially decouples rollout search width from actor-side update width. Across multiple generative recommendation tasks, ReCast consistently outperforms OpenOneRec-RL, achieving up to 36.6% relative improvement in Pass@1. Its matched-budget advantage is substantially larger: ReCast reaches the baseline's target performance with only 4.1% of the rollout budget, and this advantage widens with model scale. The same design also yields direct system-level gains, reducing actor-side update time by 16.60x, lowering peak allocated memory by 16.5%, and improving actor MFU by 14.2%. Mechanism analysis shows that ReCast mitigates the persistent all-zero / single-hit regime, restores learnability when natural positives are scarce, and converts otherwise wasted rollout budget into more stable policy updates. These results suggest that, for generative recommendation, the decisive RL problem is not only how to assign rewards, but how to construct learnable optimization events from sparse, structured supervision.

Problem

• sparse-hit
• generative recommendation
• reinforcement learning
• learnable signals
• rollout groups

Innovation

• ReCast
• generative recommendation
• sparse-hit RL
• contrastive learning signal
• learnability restoration