ShortCoder: Knowledge-Augmented Syntax Optimization for Token-Efficient Code Generation

2026-01-14

AI Summary

This work addresses the inefficiency of large language models (LLMs) in code generation, which stems from token-by-token decoding and the lack of effective optimization strategies. The authors propose a syntax-aware efficient generation approach that first introduces abstract syntax tree (AST)-preserving simplification rules for Python, enabling the construction of ShorterCodeBench—a benchmark dataset of semantically equivalent yet more concise code. Building on this, they develop a hybrid synthesis strategy combining rule-based rewriting with LLM-based refinement to achieve token-level compression. Experimental results on HumanEval demonstrate an average reduction of 18.1% in token usage, corresponding to a 18.1%–37.8% improvement in generation efficiency, while rigorously preserving functional correctness and code readability.

Technology Category

• Natural Language Processing: Code Generation / Program Synthesis from Natural Language
• Machine Learning: Large Multimodal Models (LMMs)
• Search and Optimization: Sampling/Simulation-based Search

Application Category

• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Search and Retrieval-Augmented AI: Large language models for search

Abstract

Code generation tasks aim to automate the conversion of user requirements into executable code, significantly reducing manual development efforts and enhancing software productivity. The emergence of large language models (LLMs) has significantly advanced code generation, though their efficiency is still impacted by certain inherent architectural constraints. Each token generation necessitates a complete inference pass, requiring persistent retention of contextual information in memory and escalating resource consumption. While existing research prioritizes inference-phase optimizations such as prompt compression and model quantization, the generation phase remains underexplored. To tackle these challenges, we propose a knowledge-infused framework named ShortCoder, which optimizes code generation efficiency while preserving semantic equivalence and readability. In particular, we introduce: (1) ten syntax-level simplification rules for Python, derived from AST-preserving transformations, achieving 18.1% token reduction without functional compromise; (2) a hybrid data synthesis pipeline integrating rule-based rewriting with LLM-guided refinement, producing ShorterCodeBench, a corpus of validated tuples of original code and simplified code with semantic consistency; (3) a fine-tuning strategy that injects conciseness awareness into the base LLMs. Extensive experimental results demonstrate that ShortCoder consistently outperforms state-of-the-art methods on HumanEval, achieving an improvement of 18.1%-37.8% in generation efficiency over previous methods while ensuring the performance of code generation.

Problem

• code generation
• token efficiency
• large language models
• syntax optimization
• resource consumption

Innovation

• token-efficient code generation
• syntax-level simplification
• AST-preserving transformation
• knowledge-augmented optimization
• conciseness-aware fine-tuning