Search papers, labs, and topics across Lattice.
This paper introduces ontology-based target sound extraction (TSE), a novel approach that allows sound isolation from a mixture based on semantic queries at any level of a hierarchical sound ontology. By employing a learnable class embedding table and a Cophenetic Correlation Coefficient (CPCC) loss, the model effectively aligns sound embeddings with the structural relationships in the ontology. Experimental results demonstrate significant improvements in sound extraction performance when leveraging the hierarchical organization of environmental sounds compared to traditional fixed-class systems.
Sound extraction accuracy improves dramatically when leveraging the hierarchical relationships of an ontology, allowing for flexible querying across various sound categories.
Target sound extraction (TSE) aims to isolate a sound source of interest from a mixture, given a semantic query. Existing TSE systems are conditioned on fixed class representations tied to individual sound categories, limiting their ability to handle the hierarchical relationships that naturally organize environmental sounds. In this paper, we introduce ontology-based TSE, a new task formulation in which a single model extracts sounds queried at any level of a sound ontology, from fine-grained leaf classes such as cat and dog to high-level categories such as animal. We propose a learnable class embedding table defined over all nodes of an AudioSet-derived ontology, regularized with a Cophenetic Correlation Coefficient (CPCC) loss that aligns embedding distances with shortest-path distances in the ontology tree. Our experiments across different approaches show the benefit of considering the ontology structure when training TSE systems.
