Stay Within Your Bounds: Distance-Guided Decoding for Guaranteed Context-Free Grammar Compliance

2026-08-28

AI Summary

"This study addresses the syntactic correctness of structured outputs, such as code, JSON, and SQL, generated by large language models. It proposes a forward-guided decoding framework based on pushdown automata. In the offline phase, the method computes bounded pushdown summaries with reachability labels and upper bounds on the distance to accepting states. During the online phase, these estimates guide range-aware pruning and beam search, ensuring that all outputs conform to the target grammar and effectively mitigating issues arising from tokenizer-grammar mismatches or limited token budgets. Experiments demonstrate that this approach not only maintains consistent syntactic validity across JSON, SQL, and LTL but also enhances completion quality, surpassing existing benchmarks."

Abstract

Grammar-constrained decoding helps large language models produce syntactically valid structured outputs, such as code, JSON, and SQL. For context-free grammars, many practical decoders enforce local prefix feasibility: each token must keep the current prefix extendable to some valid completion. Yet, under tokenizer-grammar mismatch and finite token budgets, feasible prefixes may still fail to reach acceptance. We propose a lookahead-guided decoding framework for context-free grammars based on pushdown automata. Offline, we compute bounded pushdown summaries with reachability labels and upper-bound distances to acceptance. Online, these estimates guide horizon-aware pruning and beam search. The resulting decoder is syntactically sound: every output is accepted by the target grammar. Experiments on JSON, SQL, and Linear Temporal Logic (LTL) show both consistent syntactic validity and improved completion quality over existing baselines.

Problem

• grammar-constrained decoding
• context-free grammars
• prefix feasibility
• tokenizer-grammar mismatch
• finite token budgets

Innovation

• lookahead-guided decoding
• pushdown automata
• context-free grammars
• horizon-aware pruning