ParaRev: Building a dataset for Scientific Paragraph Revision annotated with revision instruction

2025-01-09

AI Summary

Existing automated scientific paper revision tools operate solely at the sentence level, ignoring paragraph-level context—resulting in revisions that lack coherence and academic rigor. To address this, we formally define the paragraph-level scientific text revision task and propose a fine-grained revision instruction annotation paradigm. We introduce ParaRev, the first high-quality academic paragraph revision dataset. Methodologically, we integrate human annotation with an instruction-guided controllable generation framework, enabling training and evaluation across multiple models (e.g., T5, LLaMA). Experimental results demonstrate that our instruction-driven approach significantly outperforms general-purpose revision baselines across BLEU, BERTScore, and human evaluation metrics. These findings validate the effectiveness of paragraph-level modeling and explicit intent control in enhancing the quality of scientific writing revision.

Technology Category

• Natural Language Processing: Generation
• Cognitive Modeling & Cognitive Systems: Computational Creativity
• Machine Learning: Large Multimodal Models (LMMs)

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Revision is a crucial step in scientific writing, where authors refine their work to improve clarity, structure, and academic quality. Existing approaches to automated writing assistance often focus on sentence-level revisions, which fail to capture the broader context needed for effective modification. In this paper, we explore the impact of shifting from sentence-level to paragraph-level scope for the task of scientific text revision. The paragraph level definition of the task allows for more meaningful changes, and is guided by detailed revision instructions rather than general ones. To support this task, we introduce ParaRev, the first dataset of revised scientific paragraphs with an evaluation subset manually annotated with revision instructions. Our experiments demonstrate that using detailed instructions significantly improves the quality of automated revisions compared to general approaches, no matter the model or the metric considered.

Problem

• Scientific Paper Revision
• Contextual Editing
• Paragraph-level Analysis

Innovation

• ParaRev
• Targeted Paragraph Revision
• Concrete Revision Guidance