Unobtrusive In-Situ Measurement of Behavior Change by Deep Metric Similarity Learning of Motion Patterns

2025-09-04

AI Summary

This study addresses the challenge of non-intrusively detecting real-time behavioral changes induced by avatar morphology (e.g., height variation) in XR environments. We propose an in-situ, non-invasive measurement method based on deep metric similarity learning, which extracts high-dimensional action embeddings from head-and-hand motion trajectories and employs central-tendency analysis for personalized behavioral modeling and real-time assessment—without requiring subjective feedback. Key contributions include: (1) cross-scenario scalable analysis; (2) zero-shot adaptation to new users; and (3) precise quantification of both the Proteus Effect and body-affordance-driven behavioral differences. Experiments under multiple avatar-height conditions demonstrate the model’s high sensitivity in identifying subtle behavioral shifts, with statistically significant differences detected across diverse query-reference pairings (p < 0.01). This work establishes a novel paradigm for objective, quantitative evaluation of avatar-induced behavioral effects in XR.

Technology Category

• Cognitive Modeling & Cognitive Systems: Simulating Human Behavior
• Humans and AI: Game Design — Virtual Humans, NPCs and Autonomous Characters
• Computer Vision: Motion & Tracking

Application Category

• User Modeling, Personalization and Recommendation: Metrics for user behavior and evaluating success
• Security and Privacy: Large-scale security measurements
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web

Abstract

This paper introduces an unobtrusive in-situ measurement method to detect user behavior changes during arbitrary exposures in XR systems. Here, such behavior changes are typically associated with the Proteus effect or bodily affordances elicited by different avatars that the users embody in XR. We present a biometric user model based on deep metric similarity learning, which uses high-dimensional embeddings as reference vectors to identify behavior changes of individual users. We evaluate our model against two alternative approaches: a (non-learned) motion analysis based on central tendencies of movement patterns and subjective post-exposure embodiment questionnaires frequently used in various XR exposures. In a within-subject study, participants performed a fruit collection task while embodying avatars of different body heights (short, actual-height, and tall). Subjective assessments confirmed the effective manipulation of perceived body schema, while the (non-learned) objective analyses of head and hand movements revealed significant differences across conditions. Our similarity learning model trained on the motion data successfully identified the elicited behavior change for various query and reference data pairings of the avatar conditions. The approach has several advantages in comparison to existing methods: 1) In-situ measurement without additional user input, 2) generalizable and scalable motion analysis for various use cases, 3) user-specific analysis on the individual level, and 4) with a trained model, users can be added and evaluated in real time to study how avatar changes affect behavior.

Problem

• Detects behavior changes during XR exposures unobtrusively
• Identifies individual user behavior changes via deep metric learning
• Evaluates avatar-induced effects like Proteus effect affordances

Innovation

• Deep metric similarity learning for motion patterns
• High-dimensional embeddings as behavior reference vectors
• Real-time unobtrusive in-situ behavior change detection