ReRec: Reasoning-Augmented LLM-based Recommendation Assistant via Reinforcement Fine-tuning

2026-04-09

AI Summary

This work addresses the limited multi-step reasoning capability of current large language models in complex recommendation tasks, which often struggle to balance personalization and interpretability. To overcome this, the authors propose a reinforcement learning–based fine-tuning framework that deeply integrates structured reasoning with reinforcement learning through dual-graph–enhanced reward shaping, reasoning-aware segmented advantage estimation, and dynamic online curriculum scheduling. By combining graph neural networks, multi-objective reward modeling, and sequential action decomposition, the method significantly outperforms state-of-the-art approaches across multiple recommendation benchmarks while preserving the model’s instruction-following ability and general knowledge retention.

Abstract

With the rise of LLMs, there is an increasing need for intelligent recommendation assistants that can handle complex queries and provide personalized, reasoning-driven recommendations. LLM-based recommenders show potential but face challenges in multi-step reasoning, underscoring the need for reasoning-augmented systems. To address this gap, we propose ReRec, a novel reinforcement fine-tuning (RFT) framework designed to improve LLM reasoning in complex recommendation tasks. Our framework introduces three key components: (1) Dual-Graph Enhanced Reward Shaping, integrating recommendation metrics like NDCG@K with Query Alignment and Preference Alignment Scores to provide fine-grained reward signals for LLM optimization; (2) Reasoning-aware Advantage Estimation, which decomposes LLM outputs into reasoning segments and penalizes incorrect steps to enhance reasoning of recommendation; and (3) Online Curriculum Scheduler, dynamically assess query difficulty and organize training curriculum to ensure stable learning during RFT. Experiments demonstrate that ReRec outperforms state-of-the-art baselines and preserves core abilities like instruction-following and general knowledge. Our codes are available at https://github.com/jiani-huang/ReRec.

Problem

• LLM-based recommendation
• multi-step reasoning
• reasoning-augmented systems
• complex queries
• personalized recommendation

Innovation

• Reasoning-Augmented Recommendation
• Reinforcement Fine-tuning
• Dual-Graph Reward Shaping
• Reasoning-aware Advantage Estimation
• Online Curriculum Learning