LLM-Enhanced Commit Message Generation via Issue Information: An Exploratory Study

2026-08-22

AI Summary

研究通过结合代码差异与问题信息作为LLM输入，提出ISAC框架以改善提交信息生成，并构建ApacheCM-Issue数据集进行评估。

Abstract

Commit messages help developers understand code changes, support collaboration, and improve long-term maintenance. However, the use of issue information alone as the external context for LLM-based CMG has not been systematically studied. We propose an ISsue-Augmented framework for Commit message generation (ISAC) by combining code diffs with issue information as LLM input. To support the evaluation, we construct ApacheCM-Issue, a commit-issue aligned dataset built upon ApacheCM by linking commits with issues from GitHub and Apache Jira. Using samples from Scala, Java, and C++ projects, we evaluate four input configurations using two representative LLMs, GPT-5.5 and DeepSeek-V4-Flash in different reasoning configurations. The results show that incorporating issue information consistently improves LLM-based CMG across all evaluated model configurations and metrics, with the largest gains observed for CIDEr. Incorporating a similar historical commit further improves automatic metric scores, while replacing full issue information with a structured issue summary decreases them. ISAC also outperforms the four reproduced state-of-the-art (SOTA) CMG baselines across all five automatic metrics on the experimental dataset. The human evaluation further shows that structured issue summaries may improve perceived completeness, although replacing the original issue information can sacrifice contextual details and lead to worse results on automatic metrics.

Problem

• Commit Message Generation
• Issue Information
• LLM
• Code Changes
• Collaboration

Innovation

• LLM-based CMG
• issue information
• code diffs
• ISAC
• ApacheCM-Issue