Search papers, labs, and topics across Lattice.
This paper enhances sound event detection (SED) by introducing a semi-supervised framework that integrates conditional mixup and embedding-level contrastive loss, addressing the challenge of limited labeled data. By leveraging a combination of composition and perturbation mixup, the method effectively utilizes abundant unlabeled data during fine-tuning, leading to improved model performance. The proposed approach achieves state-of-the-art results on the DESED validation set, with scores of 0.645 PSDS1 and 0.822 PSDS2, demonstrating significant advancements in SED capabilities.
Conditional mixup bridges the gap between pseudo-labeling and contrastive learning, setting a new benchmark in sound event detection performance.
Sound event detection (SED) is a core module for acoustic environmental analysis, yet its performance is often limited by scarce labeled data. Recent systems leverage large pretrained audio foundation models, but effective fine-tuning remains challenging because labeled data are limited while unlabeled data are abundant. A previous work, ATST-SED, addressed this problem with a pseudo-label based semi-supervised fine-tuning framework. In this work, we further improve the framework by adopting an embedding-level self-supervised contrastive loss inspired by ATST-Frame pretraining. This contrastive objective better exploits unlabeled data during fine-tuning. One challenge is that mixup serves different roles in the two objectives: pseudo-label learning uses composition mixup, while contrastive learning treats mixup as a perturbation. To resolve this mismatch, we propose conditional mixup, which combines composition mixup and perturbation mixup in one semi-supervised framework and defines the corresponding embedding-level contrastive losses. The resulting model achieves 0.645 PSDS1 and 0.822 PSDS2 on the DESED validation set, establishing a new state of the art.
