Search papers, labs, and topics across Lattice.
This paper benchmarks neural speech compression through a rate-distortion lens, revealing that many existing codecs inadequately leverage probability modeling for quantized latents. The authors introduce the Entropy-Constrained Codec (ECC), which enhances low-bitrate performance by integrating learned entropy models with advanced coding techniques, including hyperprior-based side information and channel-wise context modeling. Experimental results demonstrate that ECC significantly outperforms conventional and neural codec baselines, achieving a 39.9% reduction in BD-rate on ViSQOL and 76.3% on PESQ across standard test sets.
ECC redefines low-bitrate speech compression, achieving unprecedented efficiency by integrating learned entropy modeling directly into the coding process.
Learning-based speech compression has achieved promising low-bitrate performance, but many neural speech codecs still describe quantized latents with preset-rate discrete symbols or apply entropy coding only after symbol generation. Such designs decouple representation learning from probability modeling, limiting their ability to exploit the non-uniform usage and temporal dependencies of learned speech latents. In this paper, we benchmark neural speech compression from a rate--distortion perspective and further investigate entropy-constrained coding for low-bitrate speech compression. We first formulate a unified learning-based speech coding pipeline and provide a benchmark-style analysis of recent neural speech codecs, showing that explicit probability modeling remains underexplored in learned speech compression. We then propose ECC, an Entropy-Constrained Codec that combines scalar quantization with a learned entropy model. ECC integrates hyperprior-based side information, channel-wise context modeling, latent residual prediction, and lightweight temporal modeling to estimate latent likelihoods for rate estimation during training and arithmetic coding during inference. To further improve low-bitrate efficiency, ECC introduces entropy skip, which omits highly predictable residual symbols using decoder-available scale estimates without transmitting additional skip masks. Extensive experiments show that ECC achieves a favorable low-bitrate rate--distortion trade-off over conventional and neural codec baselines, reducing BD-rate by 39.9% on ViSQOL and 76.3% on PESQ on average over two widely-used test sets. Ablation and diagnostic studies further validate the effectiveness of entropy modeling. Project Page: https://avery-xu.github.io/ECC-demo/
