hardware description languages

Current large language models (LLMs) lack standardized evaluation for hardware description language (HDL) code generation, particularly for synthesizable, functionally correct communication protocol implementations. Method: We introduce the first protocol-level RTL generation benchmark targeting SPI, I²C, UART, and AXI protocols, featuring multi-abstraction-level generation tasks and a rigorous synthesis-readiness validation pipeline—including syntax checking, logic synthesis, and UVM-driven waveform simulation. Contribution/Results: Evaluating 12 prominent open- and closed-weight LLMs, we find only two models pass all functional correctness checks, with an average synthesis success rate below 35%. Results reveal pervasive deficiencies in protocol-specific timing modeling and concurrent control handling. This benchmark fills a critical gap in evaluating LLMs for digital circuit protocol implementation and establishes a new evaluation paradigm for HDL code generation capability.

The absence of a systematic survey on large language models (LLMs) for Verilog RTL code generation hinders progress in hardware-AI co-design. Method: This paper conducts a cross-domain (software engineering/AI/EDA) analysis of 102 publications—70 peer-reviewed papers and 32 high-quality preprints—using bibliometric analysis, topic modeling, and cross-domain comparison, complemented by evaluation techniques including fine-tuning, prompt engineering, and functional correctness assessment. Contribution/Results: We introduce the first LLM-Verilog research map and a four-dimensional analytical framework addressing key research questions. Core bottlenecks are identified: architectural limitations of LLMs, scarcity of high-quality Verilog training data, and insufficient functional correctness guarantees. Furthermore, we propose a novel three-stage roadmap—“verifiable generation → hardware-aware alignment → EDA toolchain integration”—to guide future development, offering both theoretical foundations and practical pathways for LLM-driven RTL synthesis.

Hardware description language (HDL) generation faces key bottlenecks—including scarce training data, high error rates, and inefficient long-sequence modeling—hindering large language model (LLM)-driven hardware design automation. Method: This work systematically establishes high-level synthesis (HLS) as a more LLM-appropriate abstraction than low-level HDL for automated hardware design. We propose an end-to-end LLM-HLS generation framework integrating chain-of-thought prompting and iterative feedback optimization. Contribution/Results: We introduce HLSBench—the first LLM-oriented HLS benchmark—along with a dedicated fine-tuning dataset and automated evaluation pipeline. Using Llama/CodeLlama, we perform instruction tuning integrated with C++/Catapult HLS flow. Experiments demonstrate that our fine-tuned models significantly improve functional correctness and synthesizability of generated HLS code, outperforming general-purpose code models. This work establishes a new paradigm and foundational infrastructure for LLM-powered hardware design automation.

Hardware description language (HDL) understanding—particularly for VHDL in high-performance microprocessor design—lacks effective AI support. Method: This paper proposes a large language model (LLM) customization framework specifically for HDLs. Its core innovations include: (1) a novel VHDL-specific Extended Pre-Training (EPT) paradigm to enhance modeling of hardware semantics such as sequential logic and parallel constructs; (2) an LLM-as-a-judge automated evaluation framework achieving high agreement with human experts (Spearman’s ρ = 0.92); and (3) a domain-adapted benchmark suite and expert-aligned evaluation protocol. Results: Experiments show EPT improves expert-assessed accuracy from a baseline of 43% to 69%; instruction fine-tuning further raises it to 71%, with projections exceeding 85% on a newly initialized base model—significantly bridging the technical gap in intelligent VHDL comprehension.

The hardware design domain suffers from a scarcity of high-quality instruction-code pairs, reproducible benchmarks, and robust functional correctness verification mechanisms for LLM-assisted RTL design. Method: This paper introduces the first open-source dataset and benchmarking framework tailored for LLM-powered RTL design. It proposes RTLLM 2.0 (for RTL code generation) and AssertEval (for assertion generation) as dual-task benchmarks, and develops a novel RTL-simulation-based data quality filtering method to curate a 7K-sample dataset of high-confidence, functionally verified examples. Contribution/Results: Through systematic instruction-code pair construction, rigorous data cleaning, targeted model fine-tuning, and comprehensive evaluation, the framework significantly improves functional correctness in LLM-generated RTL. Experiments demonstrate that synergistic optimization of data scale, quality, and training strategy systematically enhances model performance—establishing a reproducible, verifiable infrastructure to advance LLMs in hardware design.

This work addresses the limited generalization capability of large language models (LLMs) across hardware description languages, particularly due to the absence of a systematic evaluation framework for VHDL. We propose the first unified framework for LLM-based VHDL generation and evaluation, introducing an automated, verifiable Verilog-to-VHDL benchmark conversion pipeline. The resulting VHDLBench dataset comprises over 200 VHDL modules, each accompanied by complete testbenches. Integrating automated data synthesis, the VUnit/GHDL verification toolchain, and multi-model comparative analysis, our framework enables the first comprehensive assessment of LLM-generated VHDL code in terms of compilability, executability, and functional correctness. This study reveals critical challenges posed by VHDL-specific semantics and structural constructs, laying the groundwork for multilingual hardware design automation.

This work addresses the challenge that large language models (LLMs) often introduce semantic or logical errors when generating hardware RTL code, failing to meet the stringent reliability requirements of chip design. To overcome this limitation, the paper proposes a novel hardware generation framework that integrates LLMs with formal methods, uniquely combining LLM-driven iterative refinement with formal verification. The approach leverages predefined transformation rules to guide the LLM in progressively refining high-level specifications into RTL code that is formally verifiable for correctness. This integration enhances both the interpretability and reliability of the code generation process. Experimental results demonstrate that the method is not only effective but also efficient in producing correct RTL implementations, thereby offering a promising pathway toward trustworthy LLM-assisted hardware design.

This study addresses the lack of effective multimodal evaluation benchmarks for large language models (LLMs) in hardware RTL design, which hinders accurate assessment of the functional correctness and structural plausibility of generated circuit diagrams. To bridge this gap, the authors introduce the first multimodal benchmark comprising 99 diverse RTL modules and propose a multi-stage evaluation framework that integrates rule-based scoring, self-evaluation, cross-model peer review, blind assessment, and human verification. Experimental results reveal that while current LLMs can produce visually interpretable circuit diagrams, their functional correctness remains limited. Moreover, automated evaluations exhibit almost no agreement with human judgments, highlighting the unreliability of using LLMs as evaluators in structured engineering tasks and underscoring the need for domain-specific assessment methodologies.

This work addresses the limitations of open-source Verilog/RTL code generation, which stem from the scarcity of high-quality training data and the insufficient logical reasoning and rule-expression capabilities of large language models (LLMs). To overcome these challenges, the authors propose the JRCRC pipeline, introducing a novel “Judge–Refine–Check” iterative mechanism. This approach integrates hierarchical LLMs, low-rank fine-tuning, rule-based preprocessing tools, and a dynamic logical inference module to automatically derive logical relationships from tabular data and construct a high-quality RTL training dataset. Evaluated on the VerilogEval benchmark, the method significantly outperforms existing techniques, achieving performance comparable to GPT-4o with a substantially smaller model size, thereby effectively mitigating the reasoning deficiencies of LLMs in hardware logic generation.

This work addresses the challenges of directly generating RTL code with large language models (LLMs), which often suffer from verification difficulties, limited optimizability, and poor integration with compiler-driven design flows. To overcome these issues, the authors propose the CPPL framework, which introduces a compiler-mediated interface for the first time. By leveraging a Python-based frontend DSL and a JSON-encoded CPPL intermediate representation (IR), the approach reframes LLM-assisted hardware generation as a statically checkable frontend problem. The framework exploits the CIRCT infrastructure to automatically infer bit widths, validate structural correctness, and lower designs to synthesizable Verilog. This ensures generated circuits are type-safe, hierarchically structured, verifiable, and amenable to optimization. Evaluated on the RTLLM benchmark, the method significantly improves functional correctness and, after CIRCT-based optimization, yields synthesized circuits with substantially fewer AIG nodes.