Beyond Faders: Understanding 6DoF Gesture Ecologies in Music Mixing

2026-02-26

AI Summary

This study investigates the design of six-degree-of-freedom (6DoF) gestures for music mixing in extended reality (XR) that align with professional mixing practices while minimizing cognitive load. Through design workshops, professional mix engineers contributed gesture concepts for core tasks—including gain adjustment, compression, equalization, and automation—which were subsequently implemented in an XR prototype. A user study was conducted to evaluate the ecological validity, cognitive load, and user experience of these gestures. This work presents the first systematic mapping of real-world mixing workflows into a 6DoF gesture space, enhancing users’ sense of control without compromising ecological validity. Findings indicate that the proposed gestures were widely perceived as intuitive, immersive, and well-matched to their respective tasks, with negligible differences in cognitive load; user preferences were primarily influenced by familiarity with existing workflows and perceived control.

Technology Category

• Cognitive Modeling & Cognitive Systems: Affective Computing
• Humans and AI: Interaction Techniques and Devices
• Search and Optimization: Mixed Discrete/Continuous Search

Application Category

• User Modeling, Personalization and Recommendation: Practical large-scale studies of user experience
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Economics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystems

Abstract

Extended reality (XR) enables new music-mixing workflows by moving beyond 2D faders toward embodied, spatial interaction. However, it remains unclear which six-degree-of-freedom (6DoF) gestures align with real-world mixing practices and whether such interactions support manageable cognitive load and positive user experience. We conducted a design workshop with experienced mixers to elicit gesture concepts for core audio tasks gain, compression, equalization, and automation, and implemented these in an XR prototype. A user study (n=12) evaluated the ecological validity of the gestures using cognitive load measures, user-experience ratings, and interviews. Participants generally found 6DoF gestures intuitive and well-mapped to mixing tasks, reporting strong immersion and a sense of connection with the audio environment. Cognitive load differences across gestures were minimal, though participants expressed preferences shaped by workflow familiarity and perceived control. We discuss implications for designing XR mixing tools that balance expressiveness, precision, and ecological validity.

Problem

• 6DoF gestures
• music mixing
• extended reality
• ecological validity
• cognitive load

Innovation

• 6DoF gestures
• extended reality (XR)
• music mixing
• ecological validity
• cognitive load