Search papers, labs, and topics across Lattice.
The paper introduces StyleStream, a real-time zero-shot voice style conversion system that disentangles linguistic content from style using a Destylizer and a diffusion transformer (DiT)-based Stylizer. Text supervision and a constrained information bottleneck are employed to enforce robust content-style disentanglement. The system achieves state-of-the-art performance with a fully non-autoregressive architecture, enabling real-time conversion with a 1-second end-to-end latency.
Real-time voice style conversion is now possible with StyleStream, a system that achieves state-of-the-art performance using a non-autoregressive architecture and diffusion transformers.
Voice style conversion aims to transform an input utterance to match a target speaker's timbre, accent, and emotion, with a central challenge being the disentanglement of linguistic content from style. While prior work has explored this problem, conversion quality remains limited, and real-time voice style conversion has not been addressed. We propose StyleStream, the first streamable zero-shot voice style conversion system that achieves state-of-the-art performance. StyleStream consists of two components: a Destylizer, which removes style attributes while preserving linguistic content, and a Stylizer, a diffusion transformer (DiT) that reintroduces target style conditioned on reference speech. Robust content-style disentanglement is enforced through text supervision and a highly constrained information bottleneck. This design enables a fully non-autoregressive architecture, achieving real-time voice style conversion with an end-to-end latency of 1 second. Samples and real-time demo: https://berkeley-speech-group.github.io/StyleStream/.
