DuoZone: A User-Centric, LLM-Guided Mixed-Initiative XR Window Management System

2025-11-19

AI Summary

Current XR window management relies on manual multi-application window arrangement, imposing high cognitive load and reducing operational efficiency. This paper introduces DuoZone, a hybrid proactive XR window management system that innovatively partitions the spatial workspace into a “Recommendation Zone” and an “Adjustment Zone.” It integrates LLM-driven task intent parsing, application layout recommendation, and natural spatial interactions—including voice/text input, drag-and-drop, scaling, and magnetic snapping. DuoZone supports automated layout generation under user-defined constraints while preserving direct manipulation control. A user study demonstrates that DuoZone significantly improves task completion speed (+32.7%), reduces mental workload (28.4% mean reduction in NASA-TLX scores), and enhances perceived user control (p < 0.01). Its core contribution lies in the first deep integration of large language models into a closed-loop XR window management framework, enabling semantic-aware, human-AI collaborative spatial organization.

Technology Category

• Humans and AI: Intelligent User Interfaces
• Cognitive Modeling & Cognitive Systems: Applications
• Computer Vision: Diffusion Models for Vision

Application Category

• User Modeling, Personalization and Recommendation: Practical large-scale studies of user experience
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

Mixed reality (XR) environments offer vast spatial possibilities, but current window management systems require users to manually place, resize, and organize multiple applications across large 3D spaces. This creates cognitive and interaction burdens that limit productivity. We introduce DuoZone, a mixed-initiative XR window management system that combines user-defined spatial layouts with LLM-guided automation. DuoZone separates window management into two complementary zones. The Recommendation Zone enables fast setup by providing spatial layout templates and automatically recommending relevant applications based on user tasks and high-level goals expressed through voice or text. The Arrangement Zone supports precise refinement through direct manipulation, allowing users to adjust windows using natural spatial actions such as dragging, resizing, and snapping. Through this dual-zone approach, DuoZone promotes efficient organization while reducing user cognitive load. We conducted a user study comparing DuoZone with a baseline manual XR window manager. Results show that DuoZone improves task completion speed, reduces mental effort, and increases sense of control when working with multiple applications in XR. We discuss design implications for future mixed-initiative systems and outline opportunities for integrating adaptive, goal-aware intelligence into spatial computing workflows.

Problem

• Manual XR window management creates cognitive burden in 3D spaces
• Users struggle with placing and organizing multiple applications efficiently
• Current systems lack intelligent automation for spatial layout optimization

Innovation

• Combines user-defined layouts with LLM-guided automation
• Separates window management into two complementary zones
• Uses spatial templates and natural manipulation actions