LiveFigure: Generating Editable Scientific Illustration with VLM Agents

2026-05-22

AI Summary

This work addresses the limitation of existing generative models, which produce non-editable raster images and thus fail to meet the demand in academic publishing for editable vector-based scientific illustrations. The authors propose an end-to-end framework powered by a vision-language model (VLM) agent that emulates human illustration workflows: it first plans the diagram structure by referencing high-quality exemplars, then generates executable PowerPoint scripts to render the graphics, and finally refines the output through a vision-feedback-driven correction mechanism. This approach is the first to automatically generate fully editable, publication-ready vector illustrations, enabling structured adjustments to graphical elements, proportions, attributes, and text. Experiments show that 80% of the generated illustrations meet publication standards—compared to 24% for the NanoBanana baseline—with an average of only 17 manual edits required; in human preference evaluations, the method achieves a significant 60% win rate.

Abstract

Scientific illustrations are essential for depicting conceptual designs, methodologies, and experimental workflows in research, playing a pivotal role in communicating complex academic insights. However, creating high-quality scientific illustrations remains a labor-intensive task for human scientists. While recent generative image models have advanced prompt-based editing, the synthesis of fully editable figures remains a fundamental challenge. Valid editability involves structured transformations of graphical elements, scales, attributes, and text, rather than simple pixel-level changes. Existing models generate raster outputs that do not support manual correction or layout adjustment, limiting their utility in scientific publishing, where editable vector figures are typically required for submission. To address this challenge, we introduce LiveFigure, an agentic framework driven by VLM agents that imitates the multi-step drawing workflow of human researchers. It first plans figure blueprints by drawing inspiration from high-quality references in previous works, then generates executable scripts that produce figures via the PowerPoint interface based on skills and experience, and finally refines the outputs with targeted visual diagnostics, producing fully vectorized, editable figures that meet publication standards. Extensive experiments demonstrate that LiveFigure generates inherently editable figures, achieving 80% publication-readiness in only 17 manual edits, far surpassing the 24% rate of the strongest baseline, NanoBanana. Human preference studies further validate this advantage, with LiveFigure securing a 60% win rate against NanoBanana. Our code is available at https://github.com/tsinghua-fib-lab/LiveFigure.git.

Problem

• scientific illustration
• editable figures
• vector graphics
• publication-ready
• structured editing

Innovation

• editable scientific illustration
• VLM agents
• vectorized figure generation
• agentic framework
• publication-ready figures