Merging Bodies, Dividing Conflict: Body-Swapping in Mixed Reality Increases Closeness Yet Weakens the Joint Simon Effect

2025-09-11

AI Summary

This study investigates how dyadic bodily swapping in mixed reality (MR) modulates the cognitive boundary between self and other. Method: We introduce the bodily swapping paradigm into MR for the first time, integrating motion capture with real-time virtual avatar mapping to implement an implicit measure based on the joint Simon task (JST). Contribution/Results: Bodily swapping significantly increased subjective interpersonal closeness and attenuated the joint Simon effect—indicating fusion of self- and other-representations into a unified “two-bodies-one-agent” perceptual system. These findings challenge conventional models of collaborative cognition and provide novel theoretical foundations and technical pathways for designing empathy-enhancing interventions, socially grounded learning environments, and immersive collaborative interfaces.

Technology Category

• Cognitive Modeling & Cognitive Systems: Social Cognition And Interaction
• Humans and AI: Game Design — Virtual Humans, NPCs and Autonomous Characters
• Intelligent Robots: Manipulation

Application Category

• User Modeling, Personalization and Recommendation: User modeling and simulation for interactive and conversational systems
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Economics, Online Markets and Human Computation: The sharing economy

Abstract

Mixed Reality (MR) presents novel opportunities to investigate how individuals perceive themselves and others during shared, augmented experiences within a common physical environment. Previous research has demonstrated that users can embody avatars in MR, temporarily extending their sense of self. However, there has been limited exploration of body-swapping, a condition in which two individuals simultaneously inhabit each other's avatars, and its potential effects on social interaction in immersive environments. To address this gap, we adapted the Joint Simon Task (JST), a well-established implicit paradigm, to examine how body-swapping influences the cognitive and perceptual boundaries between self and other. Our results indicate that body-swapping led participants to experience themselves and their partner as functioning like a single, unified system, as in two bodies operating as one agent. This suggests possible cognitive and perceptual changes that go beyond simple collaboration. Our findings have significant implications for the design of MR systems intended to support collaboration, empathy, social learning, and therapeutic interventions through shared embodiment.

Problem

• Examining body-swapping effects on self-other boundaries in MR
• Investigating cognitive changes during shared avatar embodiment
• Exploring how body-swapping alters social interaction dynamics

Innovation

• Adapted Joint Simon Task paradigm
• Implemented mutual avatar embodiment system
• Measured cognitive unification through body-swapping