Semi-Supervised Sound Event Detection with Conditional Mixup and Embedding-Level Contrastive Loss

2026-06-29

AI Summary

This work addresses the challenge of sound event detection under limited labeled data by proposing a semi-supervised fine-tuning framework that effectively leverages abundant unlabeled data. Building upon a pretrained audio foundation model, the approach integrates pseudo-labeling, a novel conditional mixing strategy that unifies mixup and perturbation-based augmentation, and embedding-level contrastive learning. The conditional mixing mechanism harmonizes the divergent data augmentation requirements of pseudo-label learning and contrastive learning. Evaluated on the DESED validation set, the method achieves state-of-the-art performance with PSDS1 and PSDS2 scores of 0.645 and 0.822, respectively, setting a new benchmark for sound event detection in low-resource settings.

Abstract

Sound event detection (SED) is a core module for acoustic environmental analysis, yet its performance is often limited by scarce labeled data. Recent systems leverage large pretrained audio foundation models, but effective fine-tuning remains challenging because labeled data are limited while unlabeled data are abundant. A previous work, ATST-SED, addressed this problem with a pseudo-label based semi-supervised fine-tuning framework. In this work, we further improve the framework by adopting an embedding-level self-supervised contrastive loss inspired by ATST-Frame pretraining. This contrastive objective better exploits unlabeled data during fine-tuning. One challenge is that mixup serves different roles in the two objectives: pseudo-label learning uses composition mixup, while contrastive learning treats mixup as a perturbation. To resolve this mismatch, we propose conditional mixup, which combines composition mixup and perturbation mixup in one semi-supervised framework and defines the corresponding embedding-level contrastive losses. The resulting model achieves 0.645 PSDS1 and 0.822 PSDS2 on the DESED validation set, establishing a new state of the art.

Problem

• Sound Event Detection
• Semi-Supervised Learning
• Labeled Data Scarcity
• Unlabeled Data
• Fine-tuning

Innovation

• conditional mixup
• embedding-level contrastive loss
• semi-supervised sound event detection
• pseudo-labeling
• audio foundation model fine-tuning