SciDiagramEdit: Learning to Edit Scientific Diagrams from Paper Revisions

2026-07-16

AI Summary

Editing scientific figures is a tedious process, and achieving automatic editing via natural language instructions remains highly challenging due to the heterogeneous visual elements involved and the need to adhere to strict visual grammars. This work addresses this challenge by constructing the first real-world benchmark dataset from figure revision pairs in arXiv paper submissions. It introduces a skill evolution framework that leverages editable vector representations, natural language parsing, and agent-driven multi-turn execution trajectories with feedback to iteratively refine skill descriptions and editing strategies. The approach enables collaborative figure editing between users and an intelligent agent, demonstrating consistently improved editing accuracy on a held-out validation set. These results validate that authentic revision data can effectively train high-fidelity, instruction-driven figure editing capabilities.

Abstract

Editing the figures in a research paper is a routine and time-consuming part of everyday research practice: authors relabel components, rearrange panels, and restyle visuals as they revise their manuscripts. Automating this editing workflow under a natural-language instruction, however, is challenging, because a scientific figure is a dense infographic in which heterogeneous visual elements such as schematics, plots, photos, captions, and arrows are composed under a tight visual grammar to advance a specific argument. To address this, we present SciDiagramEdit, a benchmark and skill-evolution framework that learns from natural paper revisions and operates on the figure's editable vector source, where users can inspect and co-edit individual primitives alongside the agent. Our benchmark mines before/after figure pairs from arXiv version histories, each grounded in the authors' own revision intent. To accommodate the diversity of editing instructions, we adopt agentic learning via skill evolution: an agentic proposer continually refines the agent's skill specification from execution traces over multiple epochs. The resulting skill progressively lifts edit accuracy on a held-out validation set, providing evidence that natural paper revisions are an effective training signal for instruction-driven figure editing.

Problem

• scientific diagram editing
• natural-language instruction
• vector graphics
• figure revision
• infographic automation

Innovation

• scientific diagram editing
• skill evolution
• vector graphics
• instruction-driven editing
• paper revision mining