ChipVerilog: A Large-Scale OpenCores-Derived Benchmark for LLM-Based Verilog RTL Generation

2026-07-12

AI Summary

Existing RTL generation benchmarks are largely confined to small, isolated modules and fail to capture the scale, hierarchical structure, and inter-module interaction complexity characteristic of real-world IPs or processor cores. This work introduces a large-scale Verilog RTL generation benchmark derived from OpenCores, encompassing 64 targets across five design families, including both single-module and cross-module tasks—some exceeding 1,000 lines—and accompanied by human-verified natural language descriptions and reference implementations. Through a rigorous pipeline involving natural language extraction, manual review, compilation checks, formal equivalence verification, and simulation validation, this study presents the first system-level hardware generation benchmark featuring structural complexity and genuine module dependencies. It fills a critical evaluation gap for large language models in hardware synthesis and reveals that current models still face significant challenges in generating large-scale, hierarchical RTL designs.

Abstract

Large language models have shown strong potential for Verilog RTL generation. However, many existing benchmarks are built from short, self-contained module-level tasks. These tasks are useful for controlled evaluation, but they do not fully capture the code scale, hierarchy, and module interactions found in practical IP and processor-core RTL. We present ChipVerilog, a description-to-Verilog generation benchmark built from OpenCores IP/core designs. The benchmark contains 64 generation targets from five design families: OR1200, double-precision FPU, MIPS-16, I2C, and CORDIC. It includes both single-module targets and cross-module targets that instantiate or interact with other RTL modules. Several targets exceed 1,000 lines of Verilog, making ChipVerilog substantially larger and structurally more complex than typical module-level suites. Each benchmark instance is constructed from a pair of specification documents and reference RTL. We extract the target functionality, write a detailed natural-language description, and manually review the description for correctness and clarity. Generated RTL is checked by compilation and validated through equivalence checking for local modules, or by simulation for integrated IP/core targets. Results show that large-scale RTL remains challenging, especially for hierarchical and cross-module designs.

Problem

• Verilog RTL generation
• large-scale benchmark
• hierarchical design
• module interaction
• OpenCores

Innovation

• ChipVerilog
• Verilog RTL generation
• large-scale benchmark
• hierarchical design
• cross-module interaction