Search papers, labs, and topics across Lattice.
The paper introduces P2Mark, a novel plug-and-play parameter-level watermarking method for neural speech generation (NSG) models designed for open-source scenarios. P2Mark embeds watermarks directly into the model weights using a lightweight adapter during training, enabling pre-release watermark modification and post-release security. Experiments on vocoder and codec models demonstrate that P2Mark achieves comparable performance to existing audio watermarking techniques in terms of accuracy, imperceptibility, and robustness, while providing white-box protection.
Finally, a watermarking method exists that can be embedded directly into the weights of open-source neural speech generation models, enabling proactive copyright protection.
Neural speech generation (NSG) has rapidly advanced as a key component of artificial intelligence-generated content, enabling the generation of high-quality, highly realistic speech for diverse applications. This development increases the risk of technique misuse and threatens social security. Audio watermarking can embed imperceptible marks into generated audio, providing a promising approach for secure NSG usage. However, current audio watermarking methods are mainly applied at the audio-level or feature-level, which are not suitable for open-sourced scenarios where source codes and model weights are released. To address this limitation, we propose a Plug-and-play Parameter-level WaterMarking (P2Mark) method for NSG. Specifically, we embed watermarks into the released model weights, offering a reliable solution for proactively tracing and protecting model copyrights in open-source scenarios. During training, we introduce a lightweight watermark adapter into the pre-trained model, allowing watermark information to be merged into the model via this adapter. This design ensures both the flexibility to modify the watermark before model release and the security of embedding the watermark within model parameters after model release. Meanwhile, we propose a gradient orthogonal projection optimization strategy to ensure the quality of the generated audio and the accuracy of watermark preservation. Experimental results on two mainstream waveform decoders in NSG (i.e., vocoder and codec) demonstrate that P2Mark achieves comparable performance to state-of-the-art audio watermarking methods that are not applicable to open-source white-box protection scenarios, in terms of watermark extraction accuracy, watermark imperceptibility, and robustness.
