Search papers, labs, and topics across Lattice.
This paper introduces a knowledge-driven, model-based framework for audio segmentation and source separation, leveraging external knowledge sources like music scores to guide the learning process. The approach autonomously builds hidden Markov models for audio segmentation and recognition without relying on pre-segmented training data. Experiments on both music and movie track data demonstrate that incorporating knowledge, such as music scores or sound category information, improves segmentation and source separation performance compared to data-driven techniques.
Forget painstakingly labeling training data: this method learns to segment and separate audio by leveraging external knowledge sources like music scores.
We propose a knowledge-driven, model-based approach to segmenting audio into single-category and mixed-category chunks with applications to source separation. "Knowledge" here denotes information associated with the data, such as music scores. "Model" here refers to tool that can be used for audio segmentation and recognition, such as hidden Markov models. In contrast to conventional learning that often relies on annotated data with given segment categories and their corresponding boundaries to guide the learning process, the proposed framework does not depend on any pre-segmented training data and learns directly from the input audio and its related knowledge sources to build all necessary models autonomously. Evaluation on simulation data shows that score-guided learning achieves very good music segmentation and separation results. Tested on movie track data for cinematic audio source separation also shows that utilizing sound category knowledge achieves better separation results than those obtained with data-driven techniques without using such information.
