Natural Language-Focused Software Engineering via Code-Documentation Equivalence

2026-06-20

AI Summary

Existing code documentation often suffers from incompleteness, obsolescence, or inaccuracies, hindering developer comprehension and limiting the performance of large language models (LLMs) in software engineering tasks. This work introduces, for the first time, the concept of “code-document equivalence” and presents Documentary, a novel approach that synthesizes code semantic analysis with LLM-driven natural language generation to automatically produce high-quality, semantically consistent documentation. Experimental results demonstrate that Documentary generates equivalent documentation for 53.4% of functions, significantly improving LLM accuracy on code understanding and editing tasks by 12.8–24.5% compared to both human-written and baseline documentation. Furthermore, developers consistently rate Documentary-generated documentation higher in quality and usefulness.

Abstract

Source code documentation is an integral part of software development and maintenance, as it helps in understanding the code and facilitates communication among developers. However, existing documentation is often incomplete, outdated, or inaccurate, which can lead to misunderstandings and errors. In the era of large language models (LLMs), which are being extensively used for software engineering tasks, the quality of documentation becomes even more critical, as documentation provides important context for the models. In this paper, we introduce the notion of documentation-to-code equivalence, a novel property that captures whether documentation accurately and completely describes the code it documents. We present a novel approach, called Documentary, to automatically generate equivalent documentation for a given code snippet. Our evaluation shows that Documentary can generate equivalent documentation for 53.4% of the evaluated function-level code snippets. To show the benefits of documentation-to-code equivalence, we describe and evaluate two software engineering tasks: code understanding and code editing. Our results show that documentation-to-code equivalence allows an LLM to predict the output of a function with 12.8--24.5% higher accuracy, when compared to human-written documentation and documentation generated by a baseline approach. Furthermore, human developers consider documentation generated by Documentary to be more useful for understanding and editing code than the original human-written documentation.

Problem

• code documentation
• documentation accuracy
• software maintenance
• large language models
• code understanding

Innovation

• documentation-to-code equivalence
• Documentary
• code documentation generation
• large language models
• software engineering