LC-Protonets: Multi-label Few-shot learning for world music audio tagging

2024-09-17

IEEE Open Journal of Signal Processing

AI Summary

This work addresses multi-label few-shot audio classification—specifically, automatic compound-label annotation for globally diverse world music (encompassing both traditional and contemporary genres). We propose the first prototype-based method that constructs prototypes over the power set of label combinations, rather than over individual labels, enabling zero-shot generalization to unseen label combinations without fine-tuning. Our approach integrates pretrained audio representations, models multi-source world music data, and extends prototypical networks to support multi-label prediction. Extensive experiments across cross-cultural, cross-domain, and varied few-shot settings demonstrate consistent and significant improvements over state-of-the-art methods. To ensure reproducibility and facilitate future research, we publicly release our code, a dedicated benchmark dataset, and standardized evaluation protocols. The results validate the method’s strong generalization capability, cultural adaptability, and scalability to complex, real-world multi-label audio annotation tasks.

Technology Category

• Machine Learning: Multi-class/Multi-label Learning & Extreme Classification
• Computer Vision: Multi-modal Vision
• Natural Language Processing: Language Grounding & Multi-modal NLP

Application Category

• Economics, Online Markets and Human Computation: Humans versus LLMs for data annotation and labeling
• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search
• Web Mining and Content Analysis: Mining multimedia, multimodal, multilingual, cross-lingual Web data

Abstract

We introduce Label-Combination Prototypical Networks (LC-Protonets) to address the problem of multi-label few-shot classification, where a model must generalize to new classes based on only a few available examples. Extending Prototypical Networks, LC-Protonets generate one prototype per label combination, derived from the power set of labels present in the limited training items, rather than one prototype per label. Our method is applied to automatic audio tagging across diverse music datasets, covering various cultures and including both modern and traditional music, and is evaluated against existing approaches in the literature. The results demonstrate a significant performance improvement in almost all domains and training setups when using LC-Protonets for multi-label classification. In addition to training a few-shot learning model from scratch, we explore the use of a pre-trained model, obtained via supervised learning, to embed items in the feature space. Fine-tuning improves the generalization ability of all methods, yet LC-Protonets achieve high-level performance even without fine-tuning, in contrast to the comparative approaches. We finally analyze the scalability of the proposed method, providing detailed quantitative metrics from our experiments. The implementation and experimental setup are made publicly available, offering a benchmark for future research.

Problem

• Multi-label few-shot classification
• World music audio tagging
• Prototype generation per label combination

Innovation

• Label-Combination Prototypical Networks
• Multi-label few-shot classification
• Pre-trained model fine-tuning