Search papers, labs, and topics across Lattice.
This paper explores the use of human-written album reviews from AllMusic as a complementary supervision source for training contrastive models (CLAPs) in music retrieval tasks. By preprocessing 24,5346 reviews into training-ready captions, the authors demonstrate significant improvements in retrieval performance, particularly for complex queries, compared to traditional LLM-generated captions. Additionally, the introduction of SigReg regularization enhances model performance across various classification tasks and text-to-music retrieval, leading to superior results over existing CLAP baselines.
Human-written album reviews can dramatically enhance music retrieval models, yielding substantial gains in performance for complex queries that typical datasets struggle with.
Recent open text-audio contrastive models (CLAPs) are typically trained with LLM-generated captions derived from tag datasets or web search results, which tend to be accurate but expressively narrow. As a complementary source, we explore human-written album reviews, specifically expert reviews from AllMusic: they exist at scale and carry narrative cues, evaluative adjectives, and scene framing that other sources lack. Since raw reviews are too noisy for direct use as captions, we first build a caption corpus with 24,5346 samples via an LLM preprocessing pipeline that identifies descriptive musical quotes and rewrites them into training-ready captions. We find that album review supervision yields the largest retrieval gains on a human-written caption benchmark (Song Describer), particularly for complex queries that other existing caption datasets leave uncovered. In addition, we revisit the training recipe and show that SigReg regularization, which encourages an isotropic Gaussian distribution in the embedding space, improves MLP probing across classification tasks, as well as text-to-music retrieval. The resulting model outperforms open CLAP-style baselines on text-to-music retrieval, zero-shot classification, and most MLP probing tasks. We release the review-derived caption dataset and model weights to support future research.
