CoSTAR: Data Synthesis-Driven Constraint-Aware COBOL Section Summarization for Legacy System Modernization

2026-09-10

AI Summary

为解决COBOL代码总结中的数据稀缺和迁移约束保留问题，CoSTAR框架通过执行验证的数据合成与约束感知模型训练相结合的方法来生成高质量的代码摘要。

Abstract

COBOL remains critical to governments, financial institutions, and large enterprises; yet, aging technologies, shrinking expertise, and missing documentation make modernization of COBOL-based legacy systems increasingly urgent. Before migration, code summarization is a common practice to support legacy system understanding. However, COBOL code summarization, especially on section-level, faces two key challenges: data scarcity and migration constraint preservation. To address these challenges, we propose CoSTAR, an integrated framework that combines execution-validated data synthesis with constraint-aware model training. CoSTAR repurposes general-purpose programming tasks to synthesize execution-validated COBOL code-summary data through LLM-based generation to overcome data scarcity. Based on the synthesized data, CoSTAR augments target sections with relevant data declarations and natural-language explanations, and uses constraint-guided structured rationales to train smaller base LLMs. The trained LLMs preserve the migration constraints for COBOL section summarization. We evaluate CoSTAR on both public and confidential enterprise COBOL systems. CoSTAR effectively synthesizes 3,764 execution-validated training instances. Based on these instances, CoSTAR built on 7B/8B base LLMs can improve these LLMs with average relative gains of 25.38% on ROUGE-L, 53.84% on METEOR, and 37.22% on chrF. In real-world enterprise evaluation, CoSTAR built on only Qwen3-8B, outperforms the enterprise-deployed Qwen3-235B in accuracy, completeness, and conciseness. These results show that CoSTAR enables small, locally deployable LLMs to achieve performance competitive with substantially larger LLMs for privacy-sensitive COBOL legacy systems.

Problem

• COBOL
• code summarization
• data scarcity
• migration constraints

Innovation

• Data Synthesis
• Constraint-Aware Training
• Execution-Validated Data
• COBOL Section Summarization
• Small LLMs