Problem
Research questions and friction points this paper is trying to address.
Automated Reviewing
Iterative AI Reviewing
Paper Quality Improvement
Execution-Related Weaknesses
AppliedScientist: Automated Scientific Revision Through Iterative AI Reviewing
🤖 AI Summary
本文通过结合AI科学家和AI审稿人的闭环系统，迭代修订被拒论文，以解决自动评审系统改进论文质量的问题。
📝 Abstract
Automated reviewing systems are increasingly evaluated based on the quality of the reviews they produce. Yet a review is only useful if acting on it leads to a measurable improvement in the paper. We present AppliedScientist, a closed-loop system that couples an autonomous AI scientist with an AI reviewer, and evaluate it by iteratively revising rejected papers from a range of research subfields. To mirror how human authors build on earlier drafts, the AI scientist has access to its previous versions during revision. To avoid bias from prior judgments, however, each review is generated independently, with the reviewer having no memory of earlier feedback or scores. We compare three revision settings: one initialized with the original venue reviews, one initialized with AI-generated reviews, and autonomous self-revision using the same fixed prompt in every round. Because the reviewer both guides and evaluates the revision, we also assess the human-initialized revisions using Stanford Reviewer as an independent evaluator. Reviewer-guided revision consistently improves more than fixed-prompt self-revision, and Stanford Reviewer also assigns higher scores to later revisions. AppliedScientist resolves 128 of 150 execution-related weaknesses (85.3%), but only 2 of 18 idea-related weaknesses (11.1%), suggesting that iterative revision is effective at improving experiments and implementation, but rarely changes concerns about novelty or significance.
Innovation
Methods, ideas, or system contributions that make the work stand out.
Automated Scientific Revision
Iterative AI Reviewing
Closed-Loop System
🔎 Similar Papers
No similar papers found.
