Search papers, labs, and topics across Lattice.
This study explores the impact of batch sampling strategies on text-to-audio music generation, particularly in low-data and small-scale model contexts. By clustering training data using either text or audio embeddings, the authors aim to reduce gradient interference during training. Key findings reveal that text-based clustering yields superior objective evaluation metrics, while varying cluster granularity influences both objective performance and subjective listening quality, with moderate clusters striking an optimal balance.
Clustering training data by text embeddings significantly outperforms audio embeddings in generating coherent music structures.
This work investigates the effect of batch sampling strategies during training for text-to-audio music generation under low-data and small-scale model settings. This paper describes our approach and findings for the ICME 2026 Grand Challenge on Academic Text-to-Music Generation. Training data are clustered using either text embeddings or audio embeddings, and samples with similar characteristics are grouped within the same mini-batch to mitigate gradient interference. The effects of modality and cluster granularity on clustering are analyzed. Results show that clustering based on text embeddings achieves better performance on objective evaluation metrics than clustering based on audio embeddings. In addition, different cluster granularity leads to different behaviors across evaluation criteria: a moderate number of clusters performs best on objective metrics, while a larger number of clusters tends to exhibit music with more coherent structure in listening tests.
