Search papers, labs, and topics across Lattice.
This paper introduces VibE-SVC2, an advanced singing voice conversion framework that enhances the control and performance of singing style conversion by addressing key issues from its predecessor, VibE-SVC. The model incorporates a novel Energy Style Converter to disentangle pitch and energy styles, alongside a Zero-shot Pitch Style Converter for mimicking reference audio pitch styles, and introduces vibrato rate scaling for independent vibrato control. Comprehensive evaluations reveal that VibE-SVC2 significantly outperforms existing methods, providing fine-grained control over pitch and timbre styles, including challenging phonation styles like vocal fry through a new Subharmonic Correction algorithm.
VibE-SVC2 achieves unprecedented independent control over singing styles, including pitch and timbre, while effectively addressing the challenges of pitch-energy entanglement and subharmonic characteristics.
Singing style is a crucial aspect of a natural and expressive singing voice. Singers utilize singing styles to convey the feeling or emotion of the songs. Several works have been proposed to control singing style for making the more expressive singing voice. Recently, VibE-SVC successfully controls vibrato by predicting high-frequency F0 contour. In this paper, we introduce a singing voice conversion framework, called VibE-SVC2, to improve singing style conversion performance and controllability. The model offers control over two types of singing styles: a pitch style and a timbre style. For the pitch style, to resolve the pitch-energy entanglement issue that is unresolved in our previous work, we introduce a novel Energy Style Converter to address remaining style information in the energy contour. In addition, we propose a Zero-shot Pitch Style Converter, which mimics the pitch style of reference audio. To expand the controllability of the model, we propose vibrato rate scaling that is an independent control of vibrato extent, which is unavailable in VibE-SVC. For the timbre style, we extend the model to handle a variety of phonation styles. However, addressing specific styles such as vocal fry poses a challenge, as conventional F0 extraction often fails due to their inherent subharmonic characteristics, which degrades the conversion quality. To address this, we propose a novel Subharmonic Correction algorithm to refine the F0 contour for more natural timbre conversion. Through comprehensive objective and subjective evaluations, we demonstrate that VibE-SVC2 provides fine-grained, independent control over two types of singing styles, outperforming existing methods.
