Learning to Debug: LLM-Organized Knowledge Trees for Solving RTL Assertion Failures

2025-11-21

AI Summary

In hardware verification, debugging RTL assertion failures is costly, and existing LLM-based approaches struggle to accurately model and reuse engineers’ domain-specific debugging expertise. This paper proposes GROVE, a novel framework that pioneers the organization of debugging experience into a hierarchical, LLM-learnable, and evolvable knowledge tree—enabling budget-aware iterative traversal and depth-controllable structured reasoning. GROVE employs LLM-driven gradient-free parallel training and JSON-based knowledge tree editing to jointly support knowledge navigation and hypothesis generation for precise fault localization and repair suggestions. Evaluated on an assertion-failure test suite, GROVE achieves significant improvements in pass@1 and pass@5, demonstrating the comprehensive advantages of its structured knowledge evolution mechanism in accuracy, reusability, and reasoning efficiency.

Technology Category

• Knowledge Representation and Reasoning: Diagnosis and Abductive Reasoning
• Machine Learning: Hardware-aware ML
• Search and Optimization: Learning to Search

Application Category

• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Debugging is the dominant cost in modern hardware verification, where assertion failures are among the most frequent and expensive to resolve. While Large Language Models (LLMs) show promise, they often fail to capture the precise, reusable expertise that engineers apply, leading to inaccurate responses. We propose GROVE, a hierarchical knowledge management framework that learns and organizes reusable debugging expertise into an LLM-organized knowledge tree for solving assertion failures. GROVE distills debugging knowledge from prior cases and organizes it into a vertical tree of configurable depth, with each node encoding a concise knowledge item and explicit applicability conditions. During training, GROVE uses a parallel, gradient-free loop where an LLM proposes tree modifications as structured JSON edits by learning from the cases. At test time, a budget-aware iterative zoom is performed to navigate the tree, retrieving a small set of applicable knowledge items that guide a base LLM's hypothesis generation and fix proposals. Evaluated on a suite of assertion-failure cases, GROVE delivers consistent gains in pass@1 and pass@5, demonstrating the value of structured knowledge evolution.

Problem

• Organizing reusable debugging expertise into hierarchical knowledge trees
• Improving LLM accuracy in solving RTL assertion failures
• Navigating knowledge trees for efficient hardware debugging

Innovation

• Hierarchical knowledge tree organizes debugging expertise
• Parallel gradient-free loop trains tree via JSON edits
• Budget-aware iterative zoom retrieves applicable knowledge items