VFlow: Discovering Optimal Agentic Workflows for Verilog Generation

2025-03-30

AI Summary

To address the low efficiency of generating high-quality Verilog code in hardware design automation, this paper proposes the first framework integrating Monte Carlo Tree Search (MCTS) into workflow optimization to dynamically search for optimal large language model (LLM) invocation sequences. Building upon AFLOW, we extend it with hardware-specific operators and embed a closed-loop feedback system comprising Verilog syntax checking, RTL simulation validation, and logic synthesis. This enables domain-adaptive workflow orchestration. Experimental results on the VerilogEval benchmark show a pass@1 accuracy of 83.6%, outperforming PromptV by 6.1% and direct LLM invocation by 36.9%. Moreover, our method boosts DeepSeek-V3’s performance beyond GPT-4o by 141.2%, while reducing API cost to only 13% of GPT-4o’s.

Technology Category

• Search and Optimization: Sampling/Simulation-based Search
• Machine Learning: Hardware-aware ML
• Planning, Routing, and Scheduling: Planning with Language Models

Application Category

• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information

Abstract

Hardware design automation faces challenges in generating high-quality Verilog code efficiently. This paper introduces VFlow, an automated framework that optimizes agentic workflows for Verilog code generation. Unlike existing approaches that rely on pre-defined prompting strategies, VFlow leverages Monte Carlo Tree Search (MCTS) to discover effective sequences of Large Language Models invocations that maximize code quality while minimizing computational costs. VFlow extends the AFLOW methodology with domain-specific operators addressing hardware design requirements, including syntax validation, simulation-based verification, and synthesis optimization. Experimental evaluation on the VerilogEval benchmark demonstrates VFlow's superiority, achieving an 83.6% average pass@1 rate-a 6.1% improvement over state-of-the-art PromptV and a 36.9% gain compared to direct LLM invocation. Most significantly, VFlow enhances the capabilities of smaller models, enabling DeepSeek-V3 to achieve 141.2% of GPT-4o's performance while reducing API costs to just 13%. These findings indicate that intelligently optimized workflows enable cost-efficient LLMs to outperform larger models on hardware design tasks, potentially democratizing access to advanced digital circuit development tools and accelerating innovation in the semiconductor industry

Problem

• Optimizing agentic workflows for efficient Verilog code generation
• Enhancing code quality while minimizing computational costs
• Democratizing access to advanced hardware design tools

Innovation

• Uses Monte Carlo Tree Search for optimal LLM sequences
• Extends AFLOW with hardware-specific validation operators
• Enables smaller models to outperform larger ones cost-effectively