Knots: A Large-Scale Multi-Agent Enhanced Expert-Annotated Dataset and LLM Prompt Optimization for NOTAM Semantic Parsing

2025-11-16

AI Summary

NOTAMs exhibit linguistic complexity and implicit logical structures, yet existing methods address only shallow tasks such as classification and named entity recognition, lacking deep semantic reasoning capabilities. To bridge this gap, this work introduces the novel task of NOTAM semantic parsing—emphasizing domain-specific aviation knowledge integration and structured, logic-driven output generation. We construct Knots, a large-scale expert-annotated dataset covering 194 Flight Information Regions, and propose a multi-agent collaborative data augmentation framework incorporating large language model prompting optimization and model adaptation techniques. Experimental results demonstrate significant improvements in semantic parsing accuracy. The open-sourced Knots dataset and implementation code establish a foundational resource for automated NOTAM analysis systems, advancing aviation safety information understanding from surface-level pattern recognition toward rigorous, knowledge-grounded semantic inference.

Technology Category

• Planning, Routing, and Scheduling: Planning with Language Models
• Knowledge Representation and Reasoning: Nonmonotonic Reasoning
• Natural Language Processing: Sentence-level Semantics, Textual Inference, etc.

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Vertical and domain-specific search
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets

Abstract

Notice to Air Missions (NOTAMs) serve as a critical channel for disseminating key flight safety information, yet their complex linguistic structures and implicit reasoning pose significant challenges for automated parsing. Existing research mainly focuses on surface-level tasks such as classification and named entity recognition, lacking deep semantic understanding. To address this gap, we propose NOTAM semantic parsing, a task emphasizing semantic inference and the integration of aviation domain knowledge to produce structured, inference-rich outputs. To support this task, we construct Knots (Knowledge and NOTAM Semantics), a high-quality dataset of 12,347 expert-annotated NOTAMs covering 194 Flight Information Regions, enhanced through a multi-agent collaborative framework for comprehensive field discovery. We systematically evaluate a wide range of prompt-engineering strategies and model-adaptation techniques, achieving substantial improvements in aviation text understanding and processing. Our experimental results demonstrate the effectiveness of the proposed approach and offer valuable insights for automated NOTAM analysis systems. Our code is available at: https://github.com/Estrellajer/Knots.

Problem

• Automated parsing of complex NOTAM linguistic structures for flight safety
• Addressing lack of deep semantic understanding in aviation text analysis
• Developing structured semantic parsing with aviation domain knowledge integration

Innovation

• Multi-agent framework enhances expert-annotated dataset
• Prompt optimization improves semantic parsing accuracy
• Domain knowledge integration enables structured aviation outputs