GramTrans: A Better Code Representation Approach in Code Generation

2025-10-03

AI Summary

This work investigates how code representation formats impact the code generation performance of large language models (LLMs), identifying parsing difficulty as a critical limiting factor and, for the first time, systematically validating a positive correlation between code parseability and model performance. To address this, we propose GramTrans: a method grounded in formal language theory that automatically refactors context-free grammars into LL(1)-compatible grammars, augmented by a hierarchical conflict-resolution algorithm to enable controllable trade-offs between syntactic simplicity and token efficiency. Extensive experiments across Python and Java benchmarks—using state-of-the-art models including StarCoder, DeepSeek-Coder, and Qwen2.5—demonstrate that GramTrans significantly outperforms baseline representations such as raw ASTs and S-expressions. The results confirm the method’s effectiveness, robustness, and cross-lingual generalizability.

Technology Category

• Natural Language Processing: Code Generation / Program Synthesis from Natural Language
• Machine Learning: Large Multimodal Models (LMMs)
• Planning, Routing, and Scheduling: Planning with Language Models

Application Category

• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Search and Retrieval-Augmented AI: Large language models for search

Abstract

Code generation has shown great promise in assisting software development. A fundamental yet underexplored question is how the choice of code representation affects model performance. While existing studies employ various representations, such as treating code as plain text, grammar rule sequences, or syntax tree sequences, they lack a principled understanding of the relationship between parsing difficulty and model effectiveness. This paper proposes a conjecture: the easier a representation is to parse, the better performance the model achieves. We formalize this idea using grammar classes, where representations in simpler classes (e.g., LL(1)) are easier to parse. Through a controlled experiment on a Python-based DSL, we show that parsing difficulty strongly correlates with model performance. Motivated by this finding, we present GramTrans, a general approach that automatically transforms a context-free language into a representation within the LL(1) class. GramTrans introduces a novel hierarchical conflict elimination algorithm, enabling a flexible trade-off between syntactic simplicity and token efficiency. We evaluate GramTrans on both Python and Java using three code generation models: StarCoder 1B, DeepSeek-Coder 1.3B, and Qwen2.5 1.5B. Across multiple benchmarks, GramTrans consistently delivers significant improvements over baseline representations. Furthermore, our analysis of existing representations reconfirms the strong alignment between parsing difficulty and model performance, providing additional support for the conjecture.

Problem

• Investigating how code representation choice affects model performance in generation
• Proposing easier-to-parse representations improve model effectiveness in code generation
• Developing GramTrans to transform languages into LL(1) class representations automatically

Innovation

• Transforms code into LL(1) grammar representation
• Uses hierarchical conflict elimination algorithm
• Balances syntactic simplicity with token efficiency