Toward Inferring Accurate Context-free Grammars for Big Languages in a Black-box Setting

2026-07-09

AI Summary

Existing black-box context-free grammar inference methods suffer from poor scalability, low accuracy, and limited grammatical readability when applied to large-scale languages. This work proposes Xvada, a novel deterministic inference approach that efficiently learns precise, compact, and human-readable grammars from black-box input-output observations through structured fuzzing and a rigorous empirical evaluation framework. Xvada incorporates multiple mechanisms to enhance inference quality while avoiding the evaluation biases present in prior studies. Notably, it enabled the first discovery of a CVE vulnerability in the Python Liquid engine. Compared to the state-of-the-art baseline TreeVada, Xvada achieves substantially higher grammar accuracy and compactness. Fuzzing guided by Xvada-inferred grammars uncovered six real-world vulnerabilities, all of which were confirmed and patched by developers.

Abstract

Black-box context-free grammar inference is crucial for program analysis, reverse engineering, program understanding, fuzzing, and security. But existing approaches such as Arvada, TreeVada, Kedavra, and Cucio struggle with scalability, accuracy, and grammar readability, especially on larger languages. To address this challenge, Xvada introduces several new techniques for deterministic inference of context-free grammars. In an empirical comparison that avoids several pitfalls of recent studies, Xvada improves on the highest-scoring competitor (TreeVada) both in grammar accuracy and grammar compactness. Xvada also found a CVE in the widely used Python Liquid engine. Fuzzing based on the XVada-inferred grammar found five more bugs, which the Python Liquid developers fixed based on our bug reports. XVADA and all experimental data and scripts are freely available.

Problem

• context-free grammar inference
• black-box setting
• scalability
• grammar accuracy
• large languages

Innovation

• context-free grammar inference
• black-box learning
• program analysis
• fuzzing
• grammar compactness