Adaptive 3D UI Placement in Mixed Reality Using Deep Reinforcement Learning
To address the challenge of real-time adaptive 3D UI layout under dynamic user pose and environmental changes in mixed reality (MR), this paper introduces Proximal Policy Optimization (PPO)—the first application of deep reinforcement learning to continuous 3D UI placement. We propose a multimodal state encoding mechanism that fuses user pose with scene geometry, and design a task-oriented reward function to guide the policy network toward personalized, online-adaptive layout generation. Unlike conventional optimization methods relying on static assumptions, our approach overcomes their limitations by enabling real-time responsiveness in dynamic environments. Experiments demonstrate a 23% average improvement in task completion efficiency in mobile scenarios, layout response latency under 80 ms, and significant enhancements in content visibility and interaction accessibility.