Make the Unhearable Visible: Exploring Visualization for Musical Instrument Practice.

2026-01-23

IEEE Transactions on Visualization and Computer Graphics

AI Summary

This study addresses the challenge music learners face in identifying subtle patterns and errors in their own performance due to a lack of effective feedback mechanisms. To bridge this gap, the authors propose a design philosophy of “making the inaudible visible,” integrating music signal analysis with interactive visualization techniques. Through a user-centered design process, they systematically explored 33 visualization strategies tailored to diverse instruments, skill levels, and musical attributes, and derived generalizable design principles. The development involved 18 musicians, and the resulting visualizations were evaluated by 13 learners and educators. Findings demonstrate that the proposed approach effectively supports real-time feedback and reflective practice, significantly enhancing practice efficiency, thereby offering both theoretical grounding and practical guidance for the design of intelligent music education tools.

Technology Category

• Machine Learning: Learning Preferences or Rankings
• Data Mining & Knowledge Management: Data Visualization & Summarization
• Constraint Satisfaction and Optimization: Constraint Learning and Acquisition

Application Category

• User Modeling, Personalization and Recommendation: Practical large-scale studies of user experience
• Web Mining and Content Analysis: Web data visualization
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

We explore the potential of visualization to support musicians in instrument practice through real-time feedback and reflection on their playing. Musicians often struggle to observe patterns in their playing and interpret them with respect to their goals. Our premise is that these patterns can be made visible with interactive visualization: we can make the unhearable visible. However, understanding the design of such visualizations is challenging: the diversity of needs, including different instruments, skills, musical attributes, and genres, means that any single use case is unlikely to illustrate the broad potential and opportunities. To address this challenge, we conducted a design exploration where we created and iterated on 33 designs, each focusing on a subset of needs, for example, only one musical skill. Our designs are grounded in our own experience as musicians and the ideas and feedback of 18 musicians with various musical backgrounds and we evaluated them with 13 music learners and teachers. This paper presents the results of our exploration, focusing on a few example designs as instances of possible instrument practice visualizations. From our work, we draw design considerations that contribute to future research and products for visual instrument education. Supplemental materials are available at github.com/visvar/mila.

Problem

• visualization
• musical instrument practice
• real-time feedback
• music education
• performance reflection

Innovation

• interactive visualization
• music practice
• real-time feedback
• design exploration
• musical instrument education