Search papers, labs, and topics across Lattice.
This paper introduces UniVerse, a benchmark and training solution aimed at enhancing large audio-language models (LALMs) for low-resource music understanding, particularly in culturally diverse folk music contexts. By creating UniVerseBench, a dataset of 5,042 Q&A pairs across 38 cultural entities, and UniVerseSet, a multi-turn dialogue training dataset, the authors implement imbalance-aware learning strategies to improve model performance. Experimental results show that while these methods yield significant improvements, LALMs still face challenges in capturing nuanced acoustic features, highlighting a critical gap in deep musical comprehension.
Automated data curation and imbalance-aware training strategies significantly enhance LALMs' performance on culturally diverse folk music, yet deep musical understanding remains elusive.
Recent advances in large audio-language models (LALMs) have significantly improved performance in tasks such as music captioning, genre classification, and sound event detection. However, limited attention has been paid to improving their adaptability across diverse musical traditions, particularly folk music rooted in distinct cultural contexts. Folk-music traditions are typically resource-scarce, unevenly represented across regions, and poorly documented. Even when such samples appear in large-scale pre-training, LALMs often fail to capture their structural and stylistic characteristics, partly due to the absence of dedicated evaluation protocols and training solutions. To address these limitations, we introduce UniVerse, a reproducible solution for low-resource music understanding. Specifically, we propose UniVerseBench, a benchmark of 5,042 Q&A pairs across more than 38 cultural and linguistic entities, constructed via an expert-guided yet highly automated pipeline. In parallel, we construct a fully automated, model-generated multi-turn dialogue training dataset UniVerseSet. By training LALMs on UniVerseSet, we systematically adapt and investigate representative multimodal imbalance learning strategies across both dense and Mixture-of-Experts (MoE) architectures. Experimental results indicate that fully automated data curation combined with imbalance-aware training yields non-trivial improvements, but models still struggle to capture fine-grained acoustic features, indicating a gap between surface-level alignment and deep musical comprehension.
