Search papers, labs, and topics across Lattice.
This paper introduces shape-gain decomposition into neural audio codecs (NACs) to improve robustness to variations in input signal level and enhance bitrate-distortion performance. The proposed "Equalizer" method decomposes the input signal into a gain factor and a normalized shape vector before encoding, processing the shape vector with the NAC and separately quantizing the gain. Experiments on speech signals demonstrate that this approach significantly improves bitrate-distortion performance and reduces complexity compared to joint encoding of shape and gain.
Decoupling shape and gain in neural audio codecs slashes bitrate and complexity, offering a more efficient way to encode audio.
Neural audio codecs (NACs) typically encode the short-term energy (gain) and normalized structure (shape) of speech/audio signals jointly within the same latent space. As a result, they are poorly robust to a global variation of the input signal level in the sense that such variation has strong influence on the embedding vectors at the output of the encoder and their quantization. This methodology is inherently inefficient, leading to codebook redundancy and suboptimal bitrate-distortion performance. To address these limitations, we propose to introduce shape-gain decomposition, widely used in classical speech/audio coding, into the NAC framework. The principle of the proposed Equalizer methodology is to decompose the input signal -- before the NAC encoder -- into gain and normalized shape vector on a short-term basis. The shape vector is processed by the NAC, while the gain is quantized with scalar quantization and transmitted separately. The output (decoded) signal is reconstructed from the normalized output of the NAC and the quantized gain. Our experiments conducted on speech signals show that this general methodology, easily applicable to any NAC, enables a substantial gain in bitrate-distortion performance, as well as a massive reduction in complexity.
