Evaluating ML Robustness in GNSS Interference Classification, Characterization & Localization

2024-09-23

arXiv.org

AI Summary

This study addresses the insufficient robustness of machine learning (ML) models in GNSS interference classification, characterization, and jammer localization. To this end, we introduce the first large-scale, low-frequency GNSS interference dataset comprising frequency-domain spectrogram snapshots. We systematically evaluate 129 vision encoders—including CNNs and Transformers—under realistic degradations: multipath effects, varying interference types, bandwidths, power levels, and limited input lengths. Our contributions are twofold: (1) a novel uncertainty quantification framework jointly modeling aleatoric and epistemic uncertainties to assess model adaptability to environmental shifts; and (2) a comprehensive, multi-dimensional robustness evaluation protocol tailored to real-world GNSS scenarios. Experimental results demonstrate that top-performing architectures achieve strong stability—maintaining localization error below 5 m and classification accuracy above 92%—significantly outperforming baseline methods. The dataset is publicly released to foster reproducible research.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Intelligent Robots: Localization, Mapping, and Navigation
• Computer Vision: Adversarial Attacks & Robustness

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Security and Privacy: Large-scale security measurements
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods

Abstract

Jamming devices disrupt signals from the global navigation satellite system (GNSS) and pose a significant threat, as they compromise the robustness of accurate positioning. The detection of anomalies within frequency snapshots is crucial to counteract these interferences effectively. A critical preliminary countermeasure involves the reliable classification of interferences and the characterization and localization of jamming devices. This paper introduces an extensive dataset comprising snapshots obtained from a low-frequency antenna that capture various generated interferences within a large-scale environment, including controlled multipath effects. Our objective is to assess the resilience of machine learning (ML) models against environmental changes, such as multipath effects, variations in interference attributes, such as interference class, bandwidth, and signal power, the accuracy jamming device localization, and the constraints imposed by snapshot input lengths. Furthermore, we evaluate the performance of a diverse set of 129 distinct vision encoder models across all tasks. By analyzing the aleatoric and epistemic uncertainties, we demonstrate the adaptability of our model in generalizing across diverse facets, thus establishing its suitability for real-world applications. Dataset: https://gitlab.cc-asp.fraunhofer.de/darcy_gnss/controlled_low_frequency

Problem

• Assess ML robustness in GNSS interference classification
• Characterize and localize jamming devices accurately
• Evaluate model performance in varying environmental conditions

Innovation

• Machine learning robustness evaluation
• Vision encoder models analysis
• Jamming device localization accuracy