Search papers, labs, and topics across Lattice.
This paper explores the integration of human-written album reviews from AllMusic as a complementary supervision source for training contrastive audio models (CLAPs). By preprocessing these reviews into a structured caption corpus, the authors demonstrate significant improvements in retrieval performance, especially for complex queries that existing datasets struggle with. Additionally, they introduce SigReg regularization to enhance embedding space distribution, leading to superior performance across various classification and retrieval tasks compared to existing CLAP baselines.
Human-written album reviews can dramatically enhance music retrieval models, yielding substantial gains in performance on complex queries.
Recent open text-audio contrastive models (CLAPs) are typically trained with LLM-generated captions derived from tag datasets or web search results, which tend to be accurate but expressively narrow. As a complementary source, we explore human-written album reviews, specifically expert reviews from AllMusic: they exist at scale and carry narrative cues, evaluative adjectives, and scene framing that other sources lack. Since raw reviews are too noisy for direct use as captions, we first build a caption corpus with 24,5346 samples via an LLM preprocessing pipeline that identifies descriptive musical quotes and rewrites them into training-ready captions. We find that album review supervision yields the largest retrieval gains on a human-written caption benchmark (Song Describer), particularly for complex queries that other existing caption datasets leave uncovered. In addition, we revisit the training recipe and show that SigReg regularization, which encourages an isotropic Gaussian distribution in the embedding space, improves MLP probing across classification tasks, as well as text-to-music retrieval. The resulting model outperforms open CLAP-style baselines on text-to-music retrieval, zero-shot classification, and most MLP probing tasks. We release the review-derived caption dataset and model weights to support future research.
