Customizing a Large Language Model for VHDL Design of High-Performance Microprocessors

2025-05-14

AI Summary

Hardware description language (HDL) understanding—particularly for VHDL in high-performance microprocessor design—lacks effective AI support. Method: This paper proposes a large language model (LLM) customization framework specifically for HDLs. Its core innovations include: (1) a novel VHDL-specific Extended Pre-Training (EPT) paradigm to enhance modeling of hardware semantics such as sequential logic and parallel constructs; (2) an LLM-as-a-judge automated evaluation framework achieving high agreement with human experts (Spearman’s ρ = 0.92); and (3) a domain-adapted benchmark suite and expert-aligned evaluation protocol. Results: Experiments show EPT improves expert-assessed accuracy from a baseline of 43% to 69%; instruction fine-tuning further raises it to 71%, with projections exceeding 85% on a newly initialized base model—significantly bridging the technical gap in intelligent VHDL comprehension.

Technology Category

• Machine Learning: Hardware-aware ML
• Humans and AI: Human-in-the-loop Machine Learning
• Natural Language Processing: (Large) Language Models

Application Category

• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

The use of Large Language Models (LLMs) in hardware design has taken off in recent years, principally through its incorporation in tools that increase chip designer productivity. There has been considerable discussion about the use of LLMs in RTL specifications of chip designs, for which the two most popular languages are Verilog and VHDL. LLMs and their use in Verilog design has received significant attention due to the higher popularity of the language, but little attention so far has been given to VHDL despite its continued popularity in the industry. There has also been little discussion about the unique needs of organizations that engage in high-performance processor design, and techniques to deploy AI solutions in these settings. In this paper, we describe our journey in developing a Large Language Model (LLM) specifically for the purpose of explaining VHDL code, a task that has particular importance in an organization with decades of experience and assets in high-performance processor design. We show how we developed test sets specific to our needs and used them for evaluating models as we performed extended pretraining (EPT) of a base LLM. Expert evaluation of the code explanations produced by the EPT model increased to 69% compared to a base model rating of 43%. We further show how we developed an LLM-as-a-judge to gauge models similar to expert evaluators. This led us to deriving and evaluating a host of new models, including an instruction-tuned version of the EPT model with an expected expert evaluator rating of 71%. Our experiments also indicate that with the potential use of newer base models, this rating can be pushed to 85% and beyond. We conclude with a discussion on further improving the quality of hardware design LLMs using exciting new developments in the Generative AI world.

Problem

• Developing a specialized LLM for VHDL code explanation in high-performance processor design
• Addressing the lack of AI solutions tailored for VHDL in chip design
• Improving expert-rated accuracy of LLM-generated VHDL explanations from 43% to 71%

Innovation

• Extended pretraining (EPT) of base LLM for VHDL
• Developed test sets for model evaluation
• LLM-as-a-judge to gauge model performance