Search papers, labs, and topics across Lattice.
This paper introduces LILAC, an innovative idempotent neural speech codec designed to address the limitations of existing codecs that alter audio tokens during decode-re-encode processes. By ensuring that re-encoding a decoded audio stream yields the exact original token stream, LILAC enhances the reliability of neural audio codecs in various applications, particularly in speech generation and editing pipelines. The codec operates at a competitive bitrate of 0.75 kbit/s while achieving high quality, with UTMOS scores of 4.14 and 4.24 on benchmark datasets, demonstrating its effectiveness against state-of-the-art alternatives.
LILAC ensures that re-encoding audio streams returns the original tokens, solving a critical flaw in existing neural audio codecs.
Neural Audio Codecs are widely adopted in speech generation and editing. However, existing neural audio codecs are not idempotent: across the paper's twelve baseline systems, every configuration tested rewrites, on average, at least 15% of its tokens in a single decode-re-encode pass. This poses a problem for utilizing Neural Audio Codecs as token interfaces in pipelines where re-encoding decoded outputs can occur. We present LILAC, a fully convolutional 24 kHz speech codec at 9.375 Hz and 0.75 kbit/s that is codec idempotent by construction; re-encoding the decoded audio of any valid token stream returns the identical stream. LILAC achieves idempotency while maintaining competitive quality, reaching UTMOS 4.14 and 4.24 on LibriSpeech and LibriTTS-R test sets, comparable to SOTA sub-1 kbit/s Neural Audio Codecs.
