EvaluatAR: A Cross-Device Evaluation Framework for Rapid Prototyping of Bystander PETs in AR

2026-05-27

AI Summary

Existing privacy-enhancing technologies (PETs) for AR bystanders suffer from evaluation approaches that are labor-intensive, costly, and tightly coupled to specific hardware, hindering cross-platform reproducibility and iterative development. To address this, this work proposes EvaluatAR—the first PET evaluation framework supporting multiple AR devices—enabling rapid early-stage prototyping through standardized sensor inputs and visual stimuli, along with a record-and-replay mechanism. EvaluatAR is compatible with mainstream headsets including HoloLens 2, Magic Leap 2, and Meta Quest 3, and uniformly accommodates both implicit and explicit PET designs, allowing edge cases to be replayed for algorithm diagnosis and refinement. Experimental results demonstrate that EvaluatAR not only reveals device-specific privacy-performance trade-offs and validates its cross-platform generality but also leverages replay-based optimization to enhance existing PETs beyond current state-of-the-art baselines.

Abstract

Augmented Reality (AR) headsets continuously sense their surroundings, capturing nearby bystanders and raising privacy risks. Visual bystander privacy-enhancing technologies (PETs) mitigate this risk by detecting bystanders in egocentric scene views and applying privacy transformations (e.g., obfuscation). However, traditional PET evaluation is human-dependent, high-overhead, and device-specific, making it difficult to reproduce across devices. We present EvaluatAR, a cross-device evaluation framework for rapid prototyping at the early stage of PET evaluation. Our framework enables controlled replication of experimental conditions by standardizing PET inputs (sensor data and visual stimuli) and outputs through a record-replay workflow. We validate EvaluatAR through three case studies on HoloLens 2, Magic Leap 2, and Meta Quest 3 across implicit (continuous, context-driven) and explicit (intent-driven) PETs: (1) cross-device replay of inputs to a PET to reveal device-specific privacy-performance trade-offs; (2) generalizability of the same framework workflow across implicit and explicit PET design categories; and (3) replay of privacy-relevant edge cases to diagnose failures and validate PET modifications, yielding an improvement over the state-of-the-art baseline. These results demonstrate EvaluatAR's support for rapid, iterative PET development to advance reproducible cross-device evaluation of bystander PETs at a critical moment in the emergence of ubiquitous AR.

Problem

• bystander privacy
• privacy-enhancing technologies
• cross-device evaluation
• augmented reality
• reproducibility

Innovation

• cross-device evaluation
• privacy-enhancing technologies
• augmented reality
• record-replay framework
• bystander privacy