Search papers, labs, and topics across Lattice.
This paper introduces PACodec, a low-bitrate neural speech codec that utilizes parallel additive vector quantization (PAVQ) to optimize bitrate usage by aggregating quantization results from multiple independent vector quantizers. By employing a "global-local-global" design, PACodec achieves a 30% reduction in bitrate compared to existing residual vector quantization methods while maintaining comparable decoding quality and only minor increases in model complexity. The findings indicate that the GLG framework enhances disentanglement, allowing each quantizer to focus on distinct speech aspects, which could benefit applications like voice conversion.
PACodec slashes bitrate by 30% while preserving decoding quality, thanks to its innovative parallel quantization approach.
This paper proposes PACodec, a novel low-bitrate neural speech codec based on parallel additive vector quantization (PAVQ). Unlike the mainstream residual vector quantization (RVQ) used in most neural speech codecs, where vector quantizers (VQs) are sequentially dependent, the PAVQ strategy adopted in PACodec aggregates parallel quantization results to optimize bitrate usage. Specifically, the PAVQ adopts a"global-local-global"(GLG) design: the global encoded features are quantized in parallel by multiple independent VQs, each attending to a local component of the representation, and their outputs are aggregated through addition to yield the final global quantization result for decoding. Experimental results show that PACodec, as each VQ focuses only on local information, supports smaller codebooks and reduces bitrate by 30% compared with baselines at the same decoding quality, with only minor model complexity. Further analysis shows that, owing to the GLG framework of PAVQ, the proposed PACodec is disentanglement-friendly, and each independent VQ captures different aspects of speech, e.g., content, timbre, and acoustic details, suggesting potential for application to downstream tasks such as voice conversion.
