Problem
Research questions and friction points this paper is trying to address.
meta-reviewing
scientific articles
review-rebuttal process
manuscript revisions
Metag: A dataset to build agentic meta-reviewing capabilities
🤖 AI Summary
为减轻元评审员负担，本文通过构建Metag数据集来识别论文在审稿-反驳过程中的修改，使用方法包括获取文稿版本、计算差异并由人工标注。
📝 Abstract
AI tools increasingly support tasks across the scientific research cycle, from experiment design and manuscript preparation to peer review. At the same time, the continuing growth in conference submissions has increased the burden on meta-reviewers, who must synthesize reviewer feedback, author rebuttals, and manuscript revisions. To address this concern, this paper introduces Metag, a dataset to accelerate the development of meta-reviewing agents, specifically to identify changes made to scientific articles during the review-rebuttal process. Each instance contains a reviewer concern, the author's proposed resolution, and the manuscript diffs implementing the stated change. Metag is collected by obtaining manuscript versions from before the review deadline and after acceptance, computing differences between the two documents, and asking human annotators to align these differences with action items from OpenReview discussions. The resulting dataset consists of 349 high-quality action items tied to paper differences and will enable building methods to empower meta reviewers to quickly identify whether authors have addressed reviewer statements and where in the paper those changes have been made, resulting in additional transparency and traceability throughout peer review. The dataset is publicly available at https://github.com/microsoft/Metag-dataset.
Innovation
Methods, ideas, or system contributions that make the work stand out.
Metag
meta-reviewing agents
manuscript diffs
transparency
traceability
🔎 Similar Papers
No similar papers found.
