Conversational versus Dashboard Explainable AI for UAV Intrusion Detection: An Empirical Study of Operator Trust and Reliance

2026-08-10

AI Summary

This study addresses the challenge of insufficient explainability in machine learning–based intrusion detection for unmanned aerial vehicle networks, where the black-box nature of models and high-dimensional multimodal data hinder effective operator decision-making under traditional static visualizations. The work proposes the first integration of conversational Explainable AI (XAI) into this domain, implementing an interactive interface powered by a large language model. Through a controlled user study, it investigates how such an approach influences operators’ comprehension, trust, and reliance behaviors during post-hoc auditing. Findings indicate that while conversational XAI is perceived as more useful, it may reduce operators’ self-reliance, thereby increasing the risk of over-reliance. This reveals a critical trade-off between usability and appropriate dependence in XAI interaction design and motivates a design paradigm that balances ease of use with cognitive forcing mechanisms.

Abstract

Machine learning-based Intrusion Detection Systems (IDS) have demonstrated superior performance in securing Unmanned Aerial Vehicle (UAV) networks. However, the 'black-box' nature of these models, combined with the high dimensionality of multimodal cyber-physical data, poses significant interpretability challenges. Static visualization dashboards may struggle to present complex relationships among multimodal cyber-physical features in a form that is easy for operators to inspect and interpret. To address this, we propose a Conversational XAI interface powered by Large Language Models (LLM) to facilitate on-demand investigation. In a controlled experiment with participants, we systematically evaluated the impact of this conversational interface versus a traditional XAI Dashboard on operator understanding, trust, and reliance during post-incident auditing tasks. Our results suggest that the conversational interface was perceived as more useful than the dashboard, potentially because it helped participants access and synthesize relevant information more easily. However, this benefit was accompanied by a lower level of appropriate self-reliance, indicating a potential risk of over-reliance. One possible interpretation is that the natural-language responses made the AI advice easier to accept, which may have reduced participants' tendency to verify the underlying evidence when the IDS was incorrect. These findings point to a potential trade-off in human-AI collaboration for UAV intrusion auditing: interaction mechanisms that improve perceived usability may also increase the risk of inappropriate reliance. We conclude by discussing design implications for future XAI systems that balance seamless interaction with cognitive forcing functions to foster appropriate reliance.

Problem

• Explainable AI
• UAV Intrusion Detection
• Operator Trust
• Human-AI Collaboration
• Multimodal Cyber-Physical Data

Innovation

• Conversational XAI
• Large Language Models
• UAV Intrusion Detection
• Operator Trust
• Appropriate Reliance