CircuitReason-1k: Benchmarking Long-Horizon Visual-to-Symbolic Reasoning inElectrical Circuits

2026-08-10

AI Summary

This work proposes the first long-range vision-to-symbolic reasoning benchmark framework specifically designed for electrical circuits, aiming to evaluate multimodal models’ ability to perform complete symbolic reasoning—from circuit diagram understanding through topology reconstruction, physical modeling, and equation formulation, to enforcing unit and directional consistency. The authors construct a dataset of 1,000 authentic textbook problems, each accompanied by a circuit diagram, a self-consistent question, typed answers, and reference solutions, curated via an evidence-first pipeline and organized by circuit type and dependency depth. Evaluation employs typed scoring and an identity-blinded multi-model semantic consensus mechanism to mitigate selection bias. Experiments on nine state-of-the-art multimodal models reveal a peak accuracy of 84.8%, yet performance degrades significantly in long-range reasoning, exposing systematic deficiencies in topology-goal binding, adherence to physical conventions, and reasoning propagation.

Abstract

Electrical circuit analysis requires more than recognizing components in an image. A solver must ground symbols and labels, recover latent topology, select a physical model, formulate coupled equations, propagate intermediate quantities, and preserve units, signs, directions, and phase conventions. We introduce \benchmark, a benchmark of 1,000 authentic textbook problems for evaluating this complete long-horizon visual-to-symbolic reasoning process. Each problem pairs one or more circuit diagrams with a self-contained question, a typed or semantically specified answer, and a reference worked solution. An evidence-first construction pipeline aligns questions, figures, and solutions, while a reasoning-oriented taxonomy organizes problems by circuit type and dependency depth. Evaluation combines conservative typed scoring with identity-blinded multi-model semantic consensus, retaining every problem in the denominator. Across three commercial chatbot systems and six open-source multimodal large language models, the highest-scoring system reaches 84.8\% accuracy. However, performance consistently deteriorates on long-horizon problems, and qualitative analysis exposes persistent failures in topology-to-target binding, physical conventions, and late-stage output propagation. \benchmark{} provides a focused testbed for measuring whether multimodal models can transform technical visual evidence into sustained, physically valid symbolic reasoning. Code are available at GitHub - CircuitReason/CircuitReason1K.

Problem

• visual-to-symbolic reasoning
• electrical circuits
• long-horizon reasoning
• multimodal models
• circuit analysis

Innovation

• visual-to-symbolic reasoning
• long-horizon reasoning
• electrical circuit analysis
• multimodal benchmark
• topology recovery