autogen

This work addresses the challenge of automating library API migration in the absence of real-world migration examples. To overcome this limitation, the authors propose a novel unsupervised approach that leverages large language models (LLMs) to generate initial migration examples without requiring labeled data. These examples are then generalized by an intelligent agent into structured, testable code transformation rules, which are integrated into the PolyglotPiranha framework for execution. This study represents the first integration of LLMs’ zero-shot generation capabilities with programmatic code transformation tools. The method successfully synthesizes reusable and generalizable migration scripts across multiple Python library migration tasks, significantly enhancing the feasibility and practicality of API migration in fully unsupervised settings.

This study investigates the mechanisms and effectiveness boundaries of integrating requirement information in LLM-assisted programming, addressing the fundamental limitations of current LLMs in understanding abstract, high-level requirements. Method: Through semi-structured interviews with 18 developers from 14 industrial organizations and rigorous qualitative analysis, we identify that conventional requirement documents are typically too coarse-grained and lack architectural constraints, design rationale, and contextual details necessary for reliable code generation. Effective prompting requires manual decomposition into fine-grained programming tasks enriched with such domain- and system-specific knowledge. Contribution/Results: We propose the first theoretical framework explaining requirement transformation practices in the LLM era, demonstrating that traditional requirements engineering remains indispensable in AI-augmented development. Our work not only clarifies intrinsic LLM deficiencies in semantic and contextual requirement interpretation but also establishes a foundational theoretical basis for future automated, requirement-driven software engineering paradigms.

This study addresses the unclear human-AI collaboration mechanisms in specification-driven software development with large language models (LLMs). We propose CURRANTE, a structured three-stage collaborative paradigm that guides developers through sequential refinement of requirements specifications, test cases, and function implementations. Implemented as a Visual Studio Code extension, CURRANTE integrates LLM assistance, fine-grained interaction logging, and automated test-based evaluation. By collecting interaction data and multidimensional performance metrics—including pass rates and completion time—on medium-difficulty tasks from LiveCodeBench, our work provides the first systematic empirical analysis of how iterative specification and testing dynamically influence LLM-generated code quality. These findings offer evidence-based insights for designing effective AI-augmented programming environments.

Current empirical studies on LLM-based code generation suffer from a lack of standardization, resulting in inconsistent task definitions, evaluation objectives, and metrics—severely undermining comparability and reproducibility. To address this, we propose the first systematic empirical research framework specifically designed for LLM code generation. Grounded in three core dimensions—problem source, quality attributes, and evaluation metrics—the framework synthesizes common elements from diverse empirical studies and constructs a modular, extensible assessment system via comparative analysis and theoretical modeling. Validation through representative case-study mappings demonstrates its effectiveness in enhancing experimental design rigor and reporting standardization. Our primary contribution is the establishment of the first domain-specific theoretical framework for empirical research on LLM code generation, providing foundational support for standardizing LLM evaluation in software engineering.

To address the tension between rigid template-based code generation and the high error rates of large language model (LLM)-generated code, this paper proposes iEcoreGen—a hybrid framework for LLM-augmented model-driven development. It leverages EMF/Ecore metamodels to formally specify system structure, decomposes requirements into operation-level specifications, and employs domain-specific templates to generate Java skeleton code annotated with specification-conforming docstrings; remaining method bodies are then completed by an LLM. This approach synergistically combines the correctness guarantees of model-driven engineering with the flexibility of LLMs. Evaluation across five mainstream LLMs and 20 diverse tasks shows that iEcoreGen achieves significantly higher pass@k than pure-LLM baselines, while maintaining comparable compilation@k. Ablation studies confirm the effectiveness of three core components: template-guided scaffolding, specification embedding in docstrings, and collaborative LLM-based method completion.

Large language models (LLMs) excel at function-level code generation but struggle with repository-scale system synthesis due to the ambiguity and unverifiability of natural language prompts, leading to significantly degraded output quality. To address this limitation, this work proposes Structured Specification-Driven Engineering (SSDE), a novel paradigm that, for the first time, leverages structured artifacts as inputs to guide LLMs in generating high-quality, verifiable repository-level code. The feasibility of SSDE is demonstrated through the successful automatic generation of MVC-architected business logic across three real-world software systems. These results highlight SSDE’s potential for large-scale software automation while also uncovering critical challenges and charting promising directions for future research.

This work addresses the challenges of low reliability, poor auditability, high cost, and security risks associated with runtime invocation of large language models (LLMs) in high-stakes enterprise workflows. To overcome these issues, we propose a novel “compiled AI” paradigm, wherein LLMs generate executable code during compilation, eliminating the need for model calls at runtime and thereby ensuring deterministic execution. We present the first systematic application of this paradigm to high-risk scenarios, integrating constrained code generation, a four-stage verification pipeline, template-embedded business logic functions, and an operation-oriented evaluation framework to jointly achieve reliability, auditability, and security. Experiments demonstrate a 96% success rate on function-calling tasks with zero runtime token consumption; 80.0% and 80.4% accuracy on critical field extraction and line-item recognition in document intelligence tasks; and strong security performance, with 96.7% prompt injection detection accuracy and 87.5% static analysis precision without false positives.

This work addresses the limited effectiveness of general-purpose large language models in code completion tasks within private enterprise codebases, where domain-specific structures and coding styles hinder performance. To overcome this challenge, the authors propose a semantic-scope-based approach for automatically constructing training data, which integrates retrieval-augmented generation (RAG) with supervised fine-tuning to efficiently customize medium-scale language models. Experimental results on two real-world enterprise codebases demonstrate that the resulting customized models significantly outperform larger, unadapted general-purpose models in code completion accuracy, while maintaining strong generalization capabilities on public benchmarks. These findings validate both the efficacy and practicality of the proposed method for adapting language models to proprietary software environments.

This work presents the first systematic investigation into the capability of large language models (LLMs) to generate program specifications involving higher-order logical constructs, which are essential for expressing complex verification properties yet remain beyond the reach of existing LLMs that predominantly handle basic syntactic forms. The authors design four syntactic configurations spanning different levels of abstraction and establish a comprehensive evaluation framework to assess a range of representative LLMs on standard verification benchmarks. Experimental results demonstrate that LLMs can effectively produce valid higher-order logical expressions; moreover, integrating logical constructs with base syntax significantly enhances verification efficacy and robustness without substantially increasing verification overhead. The study also reveals distinct advantages of two refinement paradigms in specification generation.