Chain-of-Descriptions: Improving Code LLMs for VHDL Code Generation and Summarization

2025-07-16

AI Summary

Current large language models (LLMs) exhibit weak performance in code generation and summarization for hardware description languages (HDLs) such as VHDL, and lack domain-specific evaluation benchmarks and methodological studies. To address this, we propose Chain-of-Descriptions (CoDes), a novel prompting paradigm that explicitly models hardware behavioral semantics by introducing multi-step functional descriptions—bridging problem specifications to VHDL code or vice versa—to enhance LLMs’ understanding and generation of functionally equivalent VHDL constructs. We evaluate CoDes on VHDL-Eval and our newly constructed VHDL-Xform dataset. Results demonstrate substantial improvements over standard prompting across generation accuracy, functional correctness, and summary readability. This work constitutes the first systematic investigation into LLM reasoning enhancement for HDLs, establishing an interpretable and scalable framework for AI-assisted hardware design.

Technology Category

• Natural Language Processing: Code Generation / Program Synthesis from Natural Language
• Machine Learning: Large Multimodal Models (LMMs)
• Computer Vision: Large Vision Models

Application Category

• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Search and Retrieval-Augmented AI: Large language models for search

Abstract

Large Language Models (LLMs) have become widely used across diverse NLP tasks and domains, demonstrating their adaptability and effectiveness. In the realm of Electronic Design Automation (EDA), LLMs show promise for tasks like Register-Transfer Level (RTL) code generation and summarization. However, despite the proliferation of LLMs for general code-related tasks, there's a dearth of research focused on evaluating and refining these models for hardware description languages (HDLs), notably VHDL. In this study, we evaluate the performance of existing code LLMs for VHDL code generation and summarization using various metrics and two datasets -- VHDL-Eval and VHDL-Xform. The latter, an in-house dataset, aims to gauge LLMs' understanding of functionally equivalent code. Our findings reveal consistent underperformance of these models across different metrics, underscoring a significant gap in their suitability for this domain. To address this challenge, we propose Chain-of-Descriptions (CoDes), a novel approach to enhance the performance of LLMs for VHDL code generation and summarization tasks. CoDes involves generating a series of intermediate descriptive steps based on: (i) the problem statement for code generation, and (ii) the VHDL code for summarization. These steps are then integrated with the original input prompt (problem statement or code) and provided as input to the LLMs to generate the final output. Our experiments demonstrate that the CoDes approach significantly surpasses the standard prompting strategy across various metrics on both datasets. This method not only improves the quality of VHDL code generation and summarization but also serves as a framework for future research aimed at enhancing code LLMs for VHDL.

Problem

• Evaluating LLMs for VHDL code generation and summarization performance
• Addressing underperformance of LLMs in hardware description languages
• Proposing Chain-of-Descriptions to improve VHDL tasks with LLMs

Innovation

• Chain-of-Descriptions enhances VHDL LLM performance
• Generates intermediate steps for code and summarization
• Integrates descriptive steps with original input prompts