A Neurosymbolic Framework for Interpretable Cognitive Attack Detection in Augmented Reality

2025-08-07

AI Summary

Existing AR cognitive attack detection methods suffer from two key limitations: vision-based approaches lack semantic reasoning capabilities, while vision-language models (VLMs) exhibit poor interpretability. To address these issues, we propose CADAR, a novel neuro-symbolic framework that integrates VLMs with symbolic reasoning for the first time. CADAR constructs a perception graph incorporating domain-specific prior knowledge, saliency-weighted visual features, and temporal associations, and employs particle filtering to enable sequential Monte Carlo inference over attack hypotheses. This establishes a closed-loop pipeline from multimodal perception to symbolic reasoning. Evaluated on an expanded AR attack dataset, CADAR achieves up to a 10.7% accuracy improvement over state-of-the-art baselines, while significantly enhancing detection transparency, interpretability, and robustness against adversarial perturbations.

Technology Category

• Computer Vision: Adversarial Attacks & Robustness
• Machine Learning: Adversarial Learning & Robustness
• Cognitive Modeling & Cognitive Systems: Symbolic Representations

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Augmented Reality (AR) enriches perception by overlaying virtual elements on the physical world. Due to its growing popularity, cognitive attacks that alter AR content to manipulate users' semantic perception have received increasing attention. Existing detection methods often focus on visual changes, which are restricted to pixel- or image-level processing and lack semantic reasoning capabilities, or they rely on pre-trained vision-language models (VLMs), which function as black-box approaches with limited interpretability. In this paper, we present CADAR, a novel neurosymbolic approach for cognitive attack detection in AR. It fuses multimodal vision-language inputs using neural VLMs to obtain a symbolic perception-graph representation, incorporating prior knowledge, salience weighting, and temporal correlations. The model then enables particle-filter based statistical reasoning -- a sequential Monte Carlo method -- to detect cognitive attacks. Thus, CADAR inherits the adaptability of pre-trained VLM and the interpretability and reasoning rigor of particle filtering. Experiments on an extended AR cognitive attack dataset show accuracy improvements of up to 10.7% over strong baselines on challenging AR attack scenarios, underscoring the promise of neurosymbolic methods for effective and interpretable cognitive attack detection.

Problem

• Detects cognitive attacks manipulating AR semantic perception
• Overcomes black-box limitations of existing vision-language models
• Combines neural adaptability with symbolic interpretability

Innovation

• Neurosymbolic fusion of vision-language inputs
• Particle-filter based statistical reasoning
• Interpretable symbolic perception-graph representation