Search papers, labs, and topics across Lattice.
This paper introduces BAMU, a novel bitstream-aware dynamic residual vector quantization (RVQ) allocation framework designed for frozen pretrained neural speech codecs, which addresses the limitations of fixed RVQ depth across varying frame complexities. By employing a lightweight, rate-independent predictor to estimate marginal latent-distortion reductions, BAMU optimally allocates quantization resources while adhering to a serialized-size budget. Experimental results demonstrate significant improvements in speech quality, with a mean opinion score (MOS) increase from 3.449 to 3.780, showcasing BAMU's effectiveness over traditional fixed-depth coding methods.
BAMU achieves a remarkable MOS improvement in speech quality by dynamically allocating quantization resources based on frame complexity, outperforming traditional fixed-depth codecs.
Pretrained neural speech codecs typically use a fixed residual vector quantization (RVQ) depth for all frames, ignoring temporal variation in quantization difficulty. We propose BAMU, a bitstream-aware dynamic RVQ allocation framework for frozen pretrained codecs. A lightweight, rate-independent predictor estimates frame- and layer-wise marginal latent-distortion reductions, while a constrained allocator selects prefix-valid depths under an exact serialized-size budget. Experiments on EnCodec and DAC over LibriSpeech, together with VCTK evaluation, show consistent EnCodec gains and DAC improvements mainly at medium and high rates. A 30-listener study confirms a MOS improvement from 3.449 to 3.780 over matched fixed-depth coding.
Tsinghua AI