R2-Write: Reflection and Revision for Open-Ended Writing with Deep Reasoning

2026-04-03

AI Summary

Existing deep reasoning models lack effective mechanisms for reflection and revision in open-ended writing tasks, hindering their ability to produce high-quality content. This work proposes R2-Write, a novel framework that explicitly integrates reflection and revision into the deep reasoning process for open-ended writing. By enabling iterative interactions between a writer and a critic, R2-Write autonomously generates structured chains of thought and incorporates a process-based reward mechanism to enhance the quality of reflective reasoning within reinforcement learning. The proposed approach significantly improves performance across multiple benchmarks for creative writing and in-depth research, while also achieving higher token efficiency. These results demonstrate both the efficacy and necessity of explicit reflection and revision mechanisms in open-ended writing tasks.

Abstract

While deep reasoning with long chain-of-thought has dramatically improved large language models in verifiable domains like mathematics, its effectiveness for open-ended tasks such as writing remains unexplored. In this paper, we conduct a systematic investigation revealing that existing mainstream reasoning models achieve limited gains on open-ended writing tasks. Our further analysis shows that these models lack deep reflection and revision patterns in open-ended writing, resulting in substantially smaller improvements compared to mathematical reasoning tasks. To address this limitation, we introduce R2-Write: an automated framework that synthesizes high-quality thinking trajectories enriched with explicit reflection and revision patterns through iterative writer-judge interaction. To prevent redundant reflections, we design a process reward mechanism that supervises reflection quality during reinforcement learning, improving both performance and token efficiency. Extensive experiments across multiple creative writing and deep-research benchmarks demonstrate significant improvements, validating that explicitly incorporating reflection and revision patterns unlocks deep reasoning capabilities for open-ended writing tasks.

Problem

• open-ended writing
• deep reasoning
• reflection
• revision
• large language models

Innovation

• reflection and revision
• deep reasoning
• open-ended writing
• process reward
• iterative writer-judge interaction