Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces MusicJudge, a novel framework for automatic singing quality assessment (SQA) that integrates both lyrical correctness and musical fidelity through a multimodal analysis approach. By employing a block-aligned method that combines semantic embeddings, lexical similarity, and phonetic alignment, MusicJudge effectively addresses the challenges of robust singing transcription, particularly in the presence of expressive variations like melisma and vibrato. Experimental results show that MusicJudge achieves strong agreement with human expert evaluations, highlighting its potential for comprehensive performance assessment in singing.
MusicJudge achieves a holistic evaluation of singing performance by seamlessly integrating lyrical and musical fidelity, outperforming traditional methods that focus on only one aspect.
Automatic singing quality assessment (SQA) requires evaluating lyrical correctness and musical fidelity while handling expressive variations. However, existing systems largely rely on either acoustic cues or lyric transcriptions exclusively, limiting holistic performance evaluation. Furthermore, their integration is non-trivial due to challenges in robust singing transcription amid melisma, vibrato, and tempo elasticity. To this end, we propose MusicJudge, a modality-guided framework for automated SQA that performs block-aligned multimodal analysis by coupling lyric correctness with pitch-rhythm fidelity. It detects semantically meaningful lyric blocks using multi-signal matching that integrates semantic embeddings, lexical similarity, and phonetic alignment. To improve singing audio transcription, we introduce Modality-Guided LoRA for ASR fine-tuning. Experiments across datasets demonstrate strong agreement with human expert judgments and validate the generalizability of MusicJudge.