Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces ContextCodec, a novel neural speech codec designed to enhance perceptual quality and intelligibility at ultra-low bitrates below 1000 bps by focusing on content-driven context features. By employing a dual-branch encoder that separates acoustic details from linguistic content and utilizing a CLIP-style contrastive loss for training, ContextCodec effectively reduces paralinguistic leakage. Experimental results demonstrate a significant improvement in quality-intelligibility trade-off down to 500 bps, achieving a real-time factor of 0.4886 on standard mobile CPUs.
ContextCodec achieves a remarkable balance between quality and intelligibility at ultra-low bitrates, outperforming existing codecs by focusing on content-driven context features.
Neural speech codecs enable low-bitrate speech communication, yet at ultra-low bitrates (< 1000 bps) preserving perceptual quality and intelligibility is challenging. Existing designs often prioritize acoustic details, leaving limited capacity for the core linguistic message under tight bitrate constraints. To address this, we propose ContextCodec, a codec that transmits content-focused context features to explicitly guide reconstruction. ContextCodec adopts a dual-branch encoder that decouples acoustic details from content-focused context. The context branch is trained with a CLIP-style contrastive loss that aligns context features with phoneme indices, reducing paralinguistic leakage. During decoding, these features are injected at each decoding stage for explicit guidance. In addition, we introduce a lightweight autoregressive latent refinement module. Experiments show a strong quality-intelligibility trade-off down to 500 bps, with an RTF of 0.4886 on a typical mobile CPU.
Tsinghua AI