Problem
Research questions and friction points this paper is trying to address.
PCB schematic understanding
spatial netlist graph
multimodal reasoning
visual grounding
diagram-to-graph conversion
OmniSch: A Multimodal PCB Schematic Benchmark For Structured Diagram Visual Reasoning
🤖 AI Summary
Current large models struggle to accurately convert PCB schematics into machine-readable, spatially weighted netlist graphs that encapsulate component attributes, connectivity relationships, and geometric information, thereby hindering advances in electronic design automation. To address this gap, this work introduces OmniSch, the first multimodal benchmark specifically designed for PCB schematic understanding and structured graph reasoning, encompassing four core tasks: visual grounding, topological relationship comprehension, geometric weight construction, and tool-augmented reasoning. Built upon a large-scale dataset of real-world schematics and integrated with semantic annotations, spatial alignment, and graph-based modeling, OmniSch establishes a fine-grained evaluation framework. Experimental results reveal significant limitations in existing large models regarding localization reliability, robustness in translating layout to graph representations, consistency in connectivity, and efficiency in visual exploration, underscoring the necessity and novelty of this benchmark.
📝 Abstract
Recent large multimodal models (LMMs) have made rapid progress in visual grounding, document understanding, and diagram reasoning tasks. However, their ability to convert Printed Circuit Board (PCB) schematic diagrams into machine-readable spatially weighted netlist graphs, jointly capturing component attributes, connectivity, and geometry, remains largely underexplored, despite such graph representations are the backbone of practical electronic design automation (EDA) workflows. To bridge this gap, we introduce OmniSch, the first comprehensive benchmark designed to assess LMMs on schematic understanding and spatial netlist graph construction. OmniSch contains 1,854 real-world schematic diagrams and includes four tasks: (1) visual grounding for schematic entities, with 109.9K grounded instances aligning 423.4K diagram semantic labels to their visual regions; (2) diagram-to-graph reasoning, understanding topological relationship among diagram elements; (3) geometric reasoning, constructing layout-dependent weights for each connection; and (4) tool-augmented agentic reasoning for visual search, invoking external tools to accomplish (1)-(3). Our results reveal substantial gaps of current LMMs in interpreting schematic engineering artifacts, including unreliable fine-grained grounding, brittle layout-to-graph parsing, inconsistent global connectivity reasoning and inefficient visual exploration.
Innovation
Methods, ideas, or system contributions that make the work stand out.
multimodal benchmark
schematic understanding
spatial netlist graph
visual grounding
diagram reasoning
🔎 Similar Papers
No similar papers found.
