Leveraging LLMs for Grammar Adaptation: A Study on Metamodel-Grammar Co-Evolution

2026-05-20

AI Summary

This work addresses the challenge of syntactic inconsistency arising from metamodel evolution, a problem poorly handled by traditional rule-based approaches in complex scenarios. For the first time, large language models (LLMs) are leveraged to enable co-evolution of metamodels and their corresponding grammars. Through prompt engineering, the approach learns grammar adaptation patterns from historical versions and automatically generates Xtext grammars aligned with updated metamodels. Evaluated on Claude Sonnet 4.5, ChatGPT 5.1, and Gemini 3 across multiple domain-specific languages, the method achieves 100% syntactic consistency and high output similarity on standard test sets, significantly outperforming rule-based techniques. Although consistency drops slightly below 90% for extremely large grammars such as EAST-ADL, the approach still demonstrates strong generalization capability and substantial potential for automation.

Abstract

In model-driven engineering, metamodel evolution leads to the need to adapt corresponding grammars to maintain consistency, which typically requires tedious manual work. Existing rule-based methods can achieve partial automation but have limitations when handling complex grammar scenarios. This paper proposes a Large Language Model-based approach that automatically applies adaptations to new grammars after evolution by learning grammar adaptations from previous versions. We evaluated this approach on six real-world Xtext domain-specific languages, using four DSLs as a training set to develop prompting strategies, two DSLs as a test set for validation, and conducting a longitudinal case study on QVTo. The evaluation used three Large Language Models (Claude Sonnet 4.5, ChatGPT 5.1, Gemini 3) and measured grammar adaptation quality from three dimensions: grammar rule-level adaptation consistency, output similarity, and metamodel conformance. Results show that on the test set, all three LLMs achieved 100% adaptation consistency and output similarity, while the rule-based approach achieved only 84.21% on DOT and 62.50% on Xcore. In the QVTo longitudinal study, the LLM-based approach successfully reused learned adaptations across all three evolution steps without manual grammar editing, while the rule-based approach required manual adjustments in two of three transitions. However, on large-scale grammars (EAST-ADL, 297 rules), LLMs' adaptation consistency was far below 90%. This study demonstrates the advantages of LLM-based approaches in handling complex grammar scenarios, while revealing their limitations in large-scale grammar adaptation.

Problem

• metamodel evolution
• grammar adaptation
• model-driven engineering
• domain-specific languages
• grammar consistency

Innovation

• Large Language Models
• Grammar Adaptation
• Metamodel Evolution
• Model-Driven Engineering
• Domain-Specific Languages