ZhuLong: Execution-Grounded LLM Agent for EDA Scripting with Offline API Self-Exploration

2026-08-08

AI Summary

This work addresses the challenges of script writing in electronic design automation (EDA), where tool-specific interfaces and insufficient documentation hinder productivity. The authors propose an execution-feedback-driven large language model agent that integrates API retrieval, documentation parsing, and sandboxed execution within a unified MCP framework. A novel offline API self-exploration mechanism, augmented with counterfactual experiments, enables the inference of undocumented API semantics without real-environment interaction. The system supports both PyAether and SKILL languages, achieving a 78.5% Pass@1 on the EDA-Eval-PyAether benchmark—54.9 percentage points higher than a pure LLM baseline. In 20 interactive tasks, it attains Pass@1 rates of 60.0% for PyAether and 50.0% for SKILL, significantly reducing tool invocations while improving script generation accuracy.

Abstract

EDA scripting with tool-specific, often undocumented APIs remains a long-tail bottleneck that existing LLMs fail to address. This paper presents ZhuLong, an execution-grounded LLM coding agent for PyAether and SKILL that combines API retrieval, documentation inspection, and sandbox execution via unified MCP tools, augmented by an offline API self-exploration mechanism that infers undocumented API behaviors through counterfactual experimentation. We evaluate ZhuLong on EDA-Eval-PyAether, a benchmark of 158 real-world tasks with assertion-based execution, where the complete system achieves 78.5% Pass@1 in the commercial Empyrean Aether environment, substantially outperforming a pure LLM baseline (23.6%). Ablation studies identify sandbox execution as the dominant performance driver (41.2 pp drop when removed), with the self-exploration mechanism contributing an additional 3.2 pp accuracy gain and a 22.1% reduction in per-task tool calls. On 20 interactive tasks involving unsaved layouts and schematics, ZhuLong achieves 60.0% Pass@1 for PyAether and 50.0% for SKILL.

Problem

• EDA scripting
• undocumented APIs
• tool-specific APIs
• LLM limitations
• long-tail bottleneck

Innovation

• execution-grounded agent
• offline API self-exploration
• counterfactual experimentation
• EDA scripting
• MCP tools