Search papers, labs, and topics across Lattice.
The paper introduces entropy-guided group residual vector quantization (EG-GRVQ) to improve neural speech codec performance at ultra-low bitrates by balancing information allocation across codebook groups. EG-GRVQ partitions the acoustic branch encoder output into groups based on channel variance, assuming Gaussian statistics to approximate information content. Experiments on LibriTTS and VCTK datasets demonstrate that EG-GRVQ enhances perceptual quality and intelligibility compared to existing methods under ultra-low bitrate constraints.
By intelligently grouping vector quantized features based on their information content, EG-GRVQ achieves state-of-the-art speech reconstruction at ultra-low bitrates.
Neural audio codec (NAC) is essential for reconstructing high-quality speech signals and generating discrete representations for downstream speech language models. However, ensuring accurate semantic modeling while maintaining high-fidelity reconstruction under ultra-low bitrate constraints remains challenging. We propose an entropy-guided group residual vector quantization (EG-GRVQ) for an ultra-low bitrate neural speech codec, which retains a semantic branch for linguistic information and incorporates an entropy-guided grouping strategy in the acoustic branch. Assuming that channel activations follow approximately Gaussian statistics, the variance of each channel can serve as a principled proxy for its information content. Based on this assumption, we partition the encoder output such that each group carries an equal share of the total information. This balanced allocation improves codebook efficiency and reduces redundancy. Trained on LibriTTS and VCTK, our model shows improvements in perceptual quality and intelligibility metrics under ultra-low bitrate conditions, with a focus on codec-level fidelity for communication-oriented scenarios.
