Large Language Model for Verilog Generation with Code-Structure-Guided Reinforcement Learning

2024-07-21

AI Summary

Current large language models (LLMs) suffer from low functional correctness and structural compliance in RTL-level Verilog code generation, primarily due to scarce Verilog training data and the inherent difficulty of modeling Verilog’s parallel syntactic structure. To address this, we propose the first reinforcement learning (RL) framework that leverages Verilog code structure as an explicit feedback signal—breaking away from the conventional pretraining-fine-tuning paradigm limited in hardware semantic modeling. Our method integrates a structure-aware RL reward function, a lightweight Verilog structural parser, and a parallelism-oriented action space for policy optimization. Evaluated on multiple Verilog code generation benchmarks, our approach significantly outperforms state-of-the-art methods: under few-shot settings, it achieves +12.7% improvement in functional correctness and +18.3% in structural compliance. This work establishes a novel, interpretable, and verifiable paradigm for hardware description language generation.

Technology Category

• Natural Language Processing: Code Generation / Program Synthesis from Natural Language
• Machine Learning: Reinforcement Learning
• Computer Vision: Large Vision Models

Application Category

• Search and Retrieval-Augmented AI: Large language models for search
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

Recent advancements in large language models (LLMs) have sparked significant interest in the automatic generation of Register Transfer Level (RTL) designs, particularly using Verilog. Current research on this topic primarily focuses on pre-training and instruction tuning, but the effectiveness of these methods is constrained by the limited availability of training data, as public Verilog code is far less abundant than software code. In particular, these methods struggle to effectively capture Verilog parallel code structures, which fundamentally differ from the imperative, sequential control flow typical in most software programming languages. This paper introduces VeriSeek, an LLM enhanced by reinforcement learning using a limited amount of high-quality training data to achieve high Verilog code generation performance. Our reinforcement learning approach employs code structure information as feedback signals to refine the pre-trained model, enabling it to effectively learn important patterns from Verilog code with parallel structures. Experiments show that VeriSeek outperforms state-of-the-art methods across multiple benchmarks.

Problem

• Enhancing Verilog generation using limited training data
• Capturing parallel code structures in Verilog effectively
• Improving LLM performance with reinforcement learning feedback

Innovation

• Reinforcement learning with code-structure feedback
• Limited high-quality data for model enhancement
• Capturing parallel Verilog structures effectively