Siren Song: Manipulating Pose Estimation in XR Headsets Using Acoustic Attacks

2025-02-13

AI Summary

This study identifies an acoustic resonance vulnerability in the inertial measurement units (IMUs) of commercial XR headsets, whereby targeted acoustic waves induce severe attitude estimation errors—up to tens of degrees—leading to virtual content misalignment and interaction hijacking. We propose the first systematic cross-layer acoustic attack framework grounded in physical resonance principles: it generates directional acoustic signals, integrates with the ORB-SLAM3 visual-inertial odometry pipeline and the ILLIXR open-source XR platform, and demonstrates end-to-end feasibility on HoloLens 2. Four novel attack primitives are realized: sensor input manipulation, click hijacking, spatial boundary intrusion, and interaction denial. Our results empirically confirm a critical acoustic side-channel threat at the physical layer of current XR systems, exposing fundamental security gaps in IMU-based tracking. This work establishes foundational insights for physical-layer security assessment and mitigation strategies in immersive computing platforms.

Technology Category

• Computer Vision: Motion & Tracking
• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Application Domains: Mobility, Driving & Flight

Application Category

• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Security and Privacy: Large-scale security measurements
• Systems and Infrastructure for Web, Mobile and WoT: Virtualization and resource management in Web systems and infrastructures

Abstract

Extended Reality (XR) experiences involve interactions between users, the real world, and virtual content. A key step to enable these experiences is the XR headset sensing and estimating the user's pose in order to accurately place and render virtual content in the real world. XR headsets use multiple sensors (e.g., cameras, inertial measurement unit) to perform pose estimation and improve its robustness, but this provides an attack surface for adversaries to interfere with the pose estimation process. In this paper, we create and study the effects of acoustic attacks that create false signals in the inertial measurement unit (IMU) on XR headsets, leading to adverse downstream effects on XR applications. We generate resonant acoustic signals on a HoloLens 2 and measure the resulting perturbations in the IMU readings, and also demonstrate both fine-grained and coarse attacks on the popular ORB-SLAM3 and an open-source XR system (ILLIXR). With the knowledge gleaned from attacking these open-source frameworks, we demonstrate four end-to-end proof-of-concept attacks on a HoloLens 2: manipulating user input, clickjacking, zone invasion, and denial of user interaction. Our experiments show that current commercial XR headsets are susceptible to acoustic attacks, raising concerns for their security.

Problem

• Acoustic attacks manipulate XR headset pose estimation.
• IMU sensor vulnerabilities exploited in XR systems.
• Demonstrated attacks compromise XR application security.

Innovation

• Acoustic attacks on IMU
• Manipulate XR pose estimation
• End-to-end proof-of-concept attacks