SciIR: A Large-scale Training Dataset and Benchmark for Scientific Image Reasoning Generation

2026-06-29

AI Summary

Existing text-to-image models struggle to meet the demands of scientific image generation, particularly in semantic alignment and logical reasoning. This work addresses this gap by introducing Peircean semiotics—specifically its triadic framework—into the task for the first time, proposing a hierarchical data construction framework tailored for scientific image reasoning. The approach formalizes scientific reasoning along three dimensions: entity structure, scientific process, and scientific law, and designs a Sci-RCoT annotation protocol to explicitly model visual logic. Leveraging this framework, the authors construct SciIR-82k, a dataset comprising 82,000 high-quality image–text pairs, and develop SciIR-Bench alongside an Atomic Checklist strategy for fine-grained evaluation. Fine-tuning Qwen-Image on this benchmark improves performance from 35% to 43%, demonstrating the efficacy of the proposed methodology.

Abstract

While Text-to-Image (T2I) models have shown remarkable success in generating photorealistic visual content, they still struggle with the rigorous semantic alignment and logical reasoning required for scientific imagery. Inspired by Peirce's Semiotic Triad, we introduce Scientific Image Reasoning (SciIR), a comprehensive resource for training and evaluation of scientific image generation. We formalize scientific reasoning into three core dimensions: Entity Structure (Icon), Scientific Process (Index), and Scientific Law (Symbol). Specifically, to overcome the scarcity of training data in scientific image generation, we elaborately create SciIR-82k, a large-scale dataset containing over 80,000 high-quality scientific image-text pairs from cutting-edge publications. The dataset is hierarchically organized according to the semiotic dimensions and incorporates a Scientific Reasoning Chain-of-Thought (Sci-RCoT) to explicitly model underlying visual logic. For evaluation, we propose SciIR-Bench, which aligns with these three semiotic levels and employs an Atomic Checklist to convert the outcome-oriented scientific accuracy into process-oriented, verifiable, fine-grained questions. Our extensive experiments reveal significant deficiencies in current models' scientific reasoning capabilities. Furthermore, by fine-tuning on the SciIR-82k dataset, we developed the Qwen-Image-SciIR model, which achieves a substantial improvement on the SciIR-Bench, increasing the final score from 35\% to 43\%, laying a solid foundation for future advances in scientific image generation.

Problem

• scientific image generation
• semantic alignment
• logical reasoning
• Text-to-Image models
• scientific reasoning

Innovation

• Scientific Image Reasoning
• Semiotic Triad
• SciIR-82k
• Scientific Reasoning Chain-of-Thought
• SciIR-Bench