Audio Prototypical Network For Controllable Music Recommendation

2025-07-31

AI Summary

Traditional music recommendation systems rely on black-box encoding models to generate user preference representations, resulting in limited interpretability and controllability—thus failing to support dynamic, fine-grained modeling of musical attributes (e.g., emotion, genre, rhythm). To address this, we introduce prototypical learning to music recommendation for the first time, proposing an interpretable user profiling method grounded in audio prototypes. Specifically, we construct perceptible and editable prototypes from semantically explicit audio content features (e.g., spectral characteristics, rhythmic patterns) to explicitly encode user preferences. Our approach enables preference cascading adjustment and interactive control. Evaluated on mainstream benchmarks, it achieves recommendation performance competitive with state-of-the-art baselines while substantially enhancing model transparency, user trust, and personalized controllability.

Technology Category

• Machine Learning: Learning Preferences or Rankings
• Knowledge Representation and Reasoning: Preferences
• Humans and AI: Learning Human Values and Preferences

Application Category

• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Economics, Online Markets and Human Computation: Economics and fairness of platforms and recommendation systems

Abstract

Traditional recommendation systems represent user preferences in dense representations obtained through black-box encoder models. While these models often provide strong recommendation performance, they lack interpretability for users, leaving users unable to understand or control the system's modeling of their preferences. This limitation is especially challenging in music recommendation, where user preferences are highly personal and often evolve based on nuanced qualities like mood, genre, tempo, or instrumentation. In this paper, we propose an audio prototypical network for controllable music recommendation. This network expresses user preferences in terms of prototypes representative of semantically meaningful features pertaining to musical qualities. We show that the model obtains competitive recommendation performance compared to popular baseline models while also providing interpretable and controllable user profiles.

Problem

• Lack of interpretability in traditional music recommendation systems
• User preferences in music are personal and nuanced
• Need for controllable and interpretable user profiles

Innovation

• Audio prototypical network for music recommendation
• User preferences expressed via semantic musical prototypes
• Interpretable and controllable user profiles