Automated Extraction of Protocol State Machines from 3GPP Specifications with Domain-Informed Prompts and LLM Ensembles

2025-10-16

AI Summary

Manual modeling of 3GPP protocol state machines is error-prone, labor-intensive, and difficult to maintain amid evolving specifications. Method: This paper proposes SpecGPT, the first framework enabling high-accuracy, automated state machine extraction from large-scale cellular protocol documents. It integrates domain-knowledge-guided prompt engineering, chain-of-thought reasoning, document segmentation, and multi-LLM ensemble strategies to address the complexity and evolutionary nature of technical standards. Contribution/Results: Evaluated on three core 5G protocols—NAS, NGAP, and PFCP—against human-annotated ground-truth datasets, SpecGPT significantly outperforms existing approaches in both accuracy and robustness. It establishes a scalable, maintainable paradigm for automated protocol modeling, directly supporting formal verification and security analysis of critical information infrastructure such as 5G networks.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Intelligent Robots: State Estimation
• Natural Language Processing: Generation

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Search and Retrieval-Augmented AI: Vertical and domain-specific search

Abstract

Mobile telecommunication networks are foundational to global infrastructure and increasingly support critical sectors such as manufacturing, transportation, and healthcare. The security and reliability of these networks are essential, yet depend heavily on accurate modeling of underlying protocols through state machines. While most prior work constructs such models manually from 3GPP specifications, this process is labor-intensive, error-prone, and difficult to maintain due to the complexity and frequent updates of the specifications. Recent efforts using natural language processing have shown promise, but remain limited in handling the scale and intricacy of cellular protocols. In this work, we propose SpecGPT, a novel framework that leverages large language models (LLMs) to automatically extract protocol state machines from 3GPP documents. SpecGPT segments technical specifications into meaningful paragraphs, applies domain-informed prompting with chain-of-thought reasoning, and employs ensemble methods to enhance output reliability. We evaluate SpecGPT on three representative 5G protocols (NAS, NGAP, and PFCP) using manually annotated ground truth, and show that it outperforms existing approaches, demonstrating the effectiveness of LLMs for protocol modeling at scale.

Problem

• Automating extraction of protocol state machines from 3GPP specifications
• Overcoming manual modeling limitations for cellular network protocols
• Enhancing reliability of automated protocol analysis using LLM ensembles

Innovation

• Uses domain-informed prompting with chain-of-thought reasoning
• Employs ensemble methods to enhance output reliability
• Segments specifications into meaningful paragraphs for processing