Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces MeloDISinger, a flow-matching-based model for text-based singing voice editing that effectively preserves melody and duration while allowing for lyric revisions. The core innovation, MeloDRP, employs phonetic cues and pseudo-MIDI melodic context to achieve precise duration control, complemented by a flow-matching mel decoder for seamless audio infilling. Experimental results indicate that MeloDISinger outperforms existing methods in both objective and subjective assessments, marking a significant advancement in singing voice editing technology.
Achieving state-of-the-art performance in singing voice editing, MeloDISinger preserves melody and duration while enabling lyric modifications with unprecedented precision.
Text-based singing voice editing (SVE) aims to revise sung lyrics while preserving the original melody, total duration, and non-edited regions. In this paper, we propose MeloDISinger, a flow-matching-based SVE model for melody-aware and duration-preserving editing. Its core module, MeloDRP, predicts fixed-budget duration ratios, enabling explicit span-wise duration control. For melody-aware duration allocation, MeloDRP fuses phonetic cues with pseudo-MIDI melodic context through cross-attention, while temporal-overlap supervision encourages soft phoneme--note correspondences. We further use a flow-matching mel decoder for audio infilling to synthesize edited regions while preserving surrounding context. In addition, we introduce a duration-aware edited-lyric generation pipeline using WhisperX and an LLM to construct feasible evaluation scenarios. Experiments demonstrate state-of-the-art performance in both objective and subjective evaluations.