Search papers, labs, and topics across Lattice.
This paper introduces a novel alignment framework to improve the musicality of lyric-to-melody generation by LLMs, which often suffer from constraint violations like poor rhythm and unsuitable vocal ranges after supervised fine-tuning. The framework uses rule-based musical constraints to automatically generate a preference dataset from an SFT model's outputs, then aligns the model using DPO followed by KTO. Experiments show the aligned model significantly reduces rule violations and outperforms baselines in objective and subjective evaluations, leading to more musical and coherent melodies.
LLMs can learn musicality without human annotation by aligning them to automatically generated preference datasets derived from rule-based musical constraints.
Large Language Models (LLMs) show promise in lyric-to-melody generation, but models trained with Supervised Fine-Tuning (SFT) often produce musically implausible melodies with issues like poor rhythm and unsuitable vocal ranges, a phenomenon we term"constraint violation". To address this, we propose a novel alignment framework that instills musical knowledge without human annotation. We define rule-based musical constraints to automatically generate a preference dataset from an SFT model's outputs. The model is then aligned through a sequential process, first using Direct Preference Optimization (DPO) on paired preference data, followed by Kahneman-Tversky Optimization (KTO) on unpaired negative samples. Experimental results demonstrate that our aligned model substantially reduces rule violations and outperforms strong baselines in both objective and subjective evaluations, generating melodies with substantially improved musicality and coherence. An interactive demo with audio comparisons is available at https://arain233.github.io/AligningMelody-demo.
