Evil Vizier: Vulnerabilities of LLM-Integrated XR Systems

2025-09-18

AI Summary

This work identifies a novel security threat arising from deep integration of extended reality (XR) and large language models (LLMs): adversaries can manipulate environmental context—such as visual or speech inputs—to induce LLMs to generate erroneous audiovisual outputs, thereby compromising user safety and privacy. To address this, the authors first systematically categorize XR-LLM integration architectures and propose a unified threat model. They empirically validate attack feasibility across diverse platforms—including Meta Quest 3, Ray-Ban Smart Glasses, HoloLens 2, and Android—using multiple LLM backends (e.g., Llama and GPT series) and frameworks. Results reveal pervasive, cross-platform context injection vulnerabilities, demonstrated via successful proof-of-concept attacks. Building on these findings, the authors design a lightweight defense prototype and provide actionable development guidelines. Collectively, this study establishes foundational theoretical insights and practical engineering pathways for building robust, secure AI-powered smart glasses.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Computer Vision: Large Vision Models
• Natural Language Processing: Safety and Robustness

Application Category

• Search and Retrieval-Augmented AI: Large language models for search
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Extended reality (XR) applications increasingly integrate Large Language Models (LLMs) to enhance user experience, scene understanding, and even generate executable XR content, and are often called "AI glasses". Despite these potential benefits, the integrated XR-LLM pipeline makes XR applications vulnerable to new forms of attacks. In this paper, we analyze LLM-Integated XR systems in the literature and in practice and categorize them along different dimensions from a systems perspective. Building on this categorization, we identify a common threat model and demonstrate a series of proof-of-concept attacks on multiple XR platforms that employ various LLM models (Meta Quest 3, Meta Ray-Ban, Android, and Microsoft HoloLens 2 running Llama and GPT models). Although these platforms each implement LLM integration differently, they share vulnerabilities where an attacker can modify the public context surrounding a legitimate LLM query, resulting in erroneous visual or auditory feedback to users, thus compromising their safety or privacy, sowing confusion, or other harmful effects. To defend against these threats, we discuss mitigation strategies and best practices for developers, including an initial defense prototype, and call on the community to develop new protection mechanisms to mitigate these risks.

Problem

• Analyzing vulnerabilities in LLM-integrated XR systems
• Demonstrating attacks causing harmful user feedback
• Proposing defenses against context manipulation threats

Innovation

• Analyzed LLM-integrated XR systems vulnerabilities
• Demonstrated attacks on multiple XR platforms
• Proposed mitigation strategies and defense prototype