Quantum Circuit Vision: Cost-Aware Evaluation of Visual AI Agents for Quantum Code Generation

2026-07-10

AI Summary

This study investigates whether multimodal AI agents can accurately generate executable code from quantum circuit diagrams and evaluates their cost-effectiveness. To this end, we introduce QCV-Dataset, the first benchmark comprising 132 quantum circuits, along with a cost-aware evaluation framework tailored for visual understanding of quantum circuits. Through systematic assessment of Claude-series multimodal models—augmented with chain-of-thought prompting, logistic regression analysis, and code validation via Amazon Braket—we identify circuit depth as the primary factor contributing to generation failures. Our experiments demonstrate that a medium-cost model achieves a 91% pass rate at only 18% of the maximum invocation cost, while a model-cascading routing strategy outperforms prompt engineering, attaining 84% accuracy at 38% cost. The dataset and implementation are publicly released.

Abstract

Can AI agents visually comprehend quantum circuit diagrams and generate verified executable code--and at what cost? We present Quantum Circuit Vision, a cost-aware evaluation framework for multimodal AI agents on quantum circuit visual understanding. We construct a 132-circuit benchmark spanning 13 categories ($1$--$10$ qubits) with executable Amazon Braket code and unitary-fidelity verification. Evaluating three frontier Claude-family models at different capability-cost tiers with $n=5$ repeated trials, we find that the mid-tier model (Sonnet 4.6, $1.30\times$ credits) offers the most favorable balance on the cost-accuracy frontier: 91% pass rate on the core subset at 18% of the per-call cost of the strongest model (Opus 4.6), whose accuracy advantage is not statistically significant (paired $t$: $p=0.083$). Logistic regression confirms that circuit depth--not qubit count--is the primary predictor of failure ($p<0.001$). Chain-of-thought prompting shows no statistically significant effect (all $p>0.18$, $n=5$), suggesting that visual pattern recognition outweighs explicit reasoning strategy for structurally coupled diagrams. We propose a cascade routing strategy (cheap $\rightarrow$ expensive models) that achieves 84% accuracy at 38% of single-model cost, demonstrating that model routing dominates prompt engineering as a cost lever. We release QCV-Dataset (132 circuits, 5 modalities, 1,931 files) on Hugging Face Hub as an open evaluation infrastructure with structured metadata for discoverability, interoperability, and responsible AI documentation, and all evaluation code, cost logs, and verification scripts on GitHub for full reproducibility.

Problem

• quantum circuit understanding
• visual AI agents
• code generation
• cost-aware evaluation
• multimodal AI

Innovation

• quantum circuit vision
• cost-aware evaluation
• multimodal AI agents
• cascade routing
• executable quantum code generation