Alignment with Fill-In-the-Middle for Enhancing Code Generation

2025-08-26

AI Summary

To address code generation under constrained settings—limited training data, scarce test cases, and difficulty in precise correctness verification—this paper proposes a verifiability-oriented LLM alignment optimization framework. Methodologically, it introduces: (1) an AST-based fine-grained code block segmentation mechanism enabling Fill-in-the-Middle-style local generation; (2) a structured segmentation–informed curriculum learning strategy for Direct Preference Optimization (DPO), yielding high-quality preference data; and (3) a phased, progressive training paradigm to enhance model sensitivity to test-driven feedback. Evaluated on five major benchmarks—HumanEval+, MBPP+, APPS, LiveCodeBench, and BigCodeBench—the framework achieves significant improvements over strong baselines. All code and data are publicly released.

Technology Category

• Natural Language Processing: Code Generation / Program Synthesis from Natural Language
• Machine Learning: Large Multimodal Models (LMMs)
• Search and Optimization: Sampling/Simulation-based Search

Application Category

• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

The code generation capabilities of Large Language Models (LLMs) have advanced applications like tool invocation and problem-solving. However, improving performance in code-related tasks remains challenging due to limited training data that is verifiable with accurate test cases. While Direct Preference Optimization (DPO) has shown promise, existing methods for generating test cases still face limitations. In this paper, we propose a novel approach that splits code snippets into smaller, granular blocks, creating more diverse DPO pairs from the same test cases. Additionally, we introduce the Abstract Syntax Tree (AST) splitting and curriculum training method to enhance the DPO training. Our approach demonstrates significant improvements in code generation tasks, as validated by experiments on benchmark datasets such as HumanEval (+), MBPP (+), APPS, LiveCodeBench, and BigCodeBench. Code and data are available at https://github.com/SenseLLM/StructureCoder.

Problem

• Improving code generation performance with limited verifiable training data
• Enhancing DPO training using AST splitting and curriculum methods
• Increasing diversity of DPO pairs through granular code splitting

Innovation

• Granular code splitting for diverse DPO pairs
• AST splitting method to enhance training
• Curriculum training approach for improved performance