Combining Textual and Spectral Features for Robust Classification of Pilot Communications

2025-09-11

AI Summary

At non-towered airports lacking surveillance infrastructure, accurately identifying aircraft operational intentions—such as takeoff or landing—from radio communications remains challenging. Method: This paper proposes the first dual-channel machine learning framework tailored to real-world aviation radio speech: it separately extracts ASR-derived textual features and Mel-spectrogram features, models them via an LSTM-CNN dual-stream architecture, fuses them through ensemble learning, and applies targeted data augmentation. Contribution/Results: To our knowledge, this is the first application of a dual-channel deep architecture to aviation intent classification. Without requiring additional hardware deployment, the framework significantly improves robustness and generalization. Experimental results demonstrate that the spectrogram channel dominates performance, achieving an F1-score of 91.3%. The approach provides a low-cost, highly scalable, and lightweight solution for automating surveillance in general aviation.

Technology Category

• Machine Learning: Multimodal Learning
• Planning, Routing, and Scheduling: Planning with Language Models
• Natural Language Processing: Speech

Application Category

• Search and Retrieval-Augmented AI: Vertical and domain-specific search
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

Accurate estimation of aircraft operations, such as takeoffs and landings, is critical for effective airport management, yet remains challenging, especially at non-towered facilities lacking dedicated surveillance infrastructure. This paper presents a novel dual pipeline machine learning framework that classifies pilot radio communications using both textual and spectral features. Audio data collected from a non-towered U.S. airport was annotated by certified pilots with operational intent labels and preprocessed through automatic speech recognition and Mel-spectrogram extraction. We evaluate a wide range of traditional classifiers and deep learning models, including ensemble methods, LSTM, and CNN across both pipelines. To our knowledge, this is the first system to classify operational aircraft intent using a dual-pipeline ML framework on real-world air traffic audio. Our results demonstrate that spectral features combined with deep architectures consistently yield superior classification performance, with F1-scores exceeding 91%. Data augmentation further improves robustness to real-world audio variability. The proposed approach is scalable, cost-effective, and deployable without additional infrastructure, offering a practical solution for air traffic monitoring at general aviation airports.

Problem

• Classifying pilot radio communications using textual and spectral features
• Estimating aircraft operations at non-towered airports lacking surveillance
• Developing dual-pipeline ML framework for real-world air traffic audio

Innovation

• Dual pipeline ML framework
• Combines textual and spectral features
• Data augmentation improves robustness