🤖 AI Summary
This work addresses the challenge that large language models face in generating syntactically valid structured outputs—such as JSON—under strict token-length constraints, often resulting in either truncated invalid sequences or unbounded generation. To resolve this, the authors propose TruncProof, a novel approach that integrates LL(1) grammar parsing with token-length awareness during decoding. At each generation step, TruncProof dynamically estimates the minimum number of tokens required to complete a syntactically valid JSON object and incorporates this constraint into advanced decoding strategies such as beam search. The method achieves 100% syntactic correctness in generated JSON under stringent length limits while preserving high semantic fidelity, thereby overcoming a key limitation of existing techniques that fail to simultaneously ensure structural validity and controllable output length.
📝 Abstract
The LLM-based generation of machine-readable outputs such as JSON has attracted significant attention for integration with external systems. However, existing approaches cannot strictly enforce the maximum number of tokens to be generated, leading to infinite generation or truncated outputs that cause a system malfunction. To address this limitation, we propose TruncProof, a novel grammar-constrained generation method that enables LLMs to produce grammatically valid JSONs while adhering to a predefined token limit. By leveraging the properties of LL(1) parsers, TruncProof efficiently approximates the minimum number of tokens required to complete a grammatically valid output at each decoding step. Experiments on the Text-to-JSON instruction tasks demonstrate that TruncProof successfully generates syntactically correct outputs even under strict token constraints. Furthermore, we show that TruncProof can be effectively combined with advanced decoding strategies, resulting in outputs that are not only grammatically valid but also semantically accurate.