Search papers, labs, and topics across Lattice.
This paper introduces Elastic Time, a dynamic frame-rate bottleneck for neural audio autoencoders that optimizes temporal resolution by allowing variable frame rates based on information density. By employing a lightweight latent predictor, the method intelligently determines which frames can be skipped during encoding, enhancing efficiency without sacrificing quality. Experimental results demonstrate that Elastic Time significantly improves the efficiency-quality tradeoff compared to traditional fixed-frame-rate approaches, paving the way for more adaptable audio processing models.
Dynamic frame rates in audio autoencoders can drastically enhance efficiency, allowing for smarter resource allocation in neural compression tasks.
Neural audio autoencoders have become a core component of compression, feature extraction, and generation. However, while existing systems support variable bitrate, the vast majority of models still operate at a fixed latent frame-rate, allocating equal temporal budget to regions with very different information density, which can result in unnecessarily long sequences. We introduce Elastic Time, a dynamic frame-rate bottleneck that converts fixed-frame-rate autoencoders to dynamic ones. Our method learns a lightweight latent predictor used to decide which frames can be skipped and later reconstructed, enabling efficient greedy boundary selection at inference. Experiments show our method enables deployment-time rate control while improving efficiency-quality tradeoffs relative to baselines. Overall, we provide a flexible mechanism for adjusting temporal resolution in audio autoencoders, potentially facilitating more efficient downstream modeling for generation and long-context tasks.
MIT CSAIL