Insights from Rights and Wrongs: A Large Language Model for Solving Assertion Failures in RTL Design

2025-03-06

AI Summary

In RTL design, root-cause analysis and repair of SystemVerilog Assertion (SVA) failures heavily rely on expert knowledge, with minimal automation support. This paper introduces AssertSolver—the first open-source domain-specific large language model (DS-LLM) tailored for SVA debugging. Our method comprises two key innovations: (1) a domain-knowledge-enhanced architecture integrating RTL semantics, assertion logic, and temporal relationships; and (2) an error-driven synthetic data generation and reinforcement fine-tuning paradigm, significantly improving attribution accuracy and fix generation for assertion violations. Evaluated on a novel, comprehensive benchmark we curate, AssertSolver achieves 88.54% pass@1 bug-fixing accuracy—outperforming OpenAI o1-preview by 11.97%. To foster reproducibility and community advancement, we fully open-source the model, training data, evaluation benchmark, and implementation code.

Technology Category

• Knowledge Representation and Reasoning: Diagnosis and Abductive Reasoning
• Constraint Satisfaction and Optimization: Satisfiability Modulo Theories
• Machine Learning: Large Multimodal Models (LMMs)

Application Category

• Search and Retrieval-Augmented AI: Vertical and domain-specific search
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

SystemVerilog Assertions (SVAs) are essential for verifying Register Transfer Level (RTL) designs, as they can be embedded into key functional paths to detect unintended behaviours. During simulation, assertion failures occur when the design's behaviour deviates from expectations. Solving these failures, i.e., identifying and fixing the issues causing the deviation, requires analysing complex logical and timing relationships between multiple signals. This process heavily relies on human expertise, and there is currently no automatic tool available to assist with it. Here, we present AssertSolver, an open-source Large Language Model (LLM) specifically designed for solving assertion failures. By leveraging synthetic training data and learning from error responses to challenging cases, AssertSolver achieves a bug-fixing pass@1 metric of 88.54% on our testbench, significantly outperforming OpenAI's o1-preview by up to 11.97%. We release our model and testbench for public access to encourage further research: https://github.com/SEU-ACAL/reproduce-AssertSolver-DAC-25.

Problem

• Automates solving assertion failures in RTL designs
• Identifies and fixes issues causing behavioral deviations
• Reduces reliance on human expertise in debugging SVAs

Innovation

• LLM for solving RTL assertion failures
• Synthetic data enhances bug-fixing accuracy
• Open-source model outperforms existing solutions