Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces DTM-Codec, a neural speech codec that employs Dynamic Token Masking to enhance variable frame rate (VFR) coding by efficiently selecting which tokens to retain and transmitting a binary keep-mask for decoding. The method significantly outperforms fixed-frame-rate baselines in terms of reconstruction quality and intelligibility while adhering to a strict matched-total-bitrate protocol. Additionally, the authors present Path Length Equalization (PLE), a novel boundary selection technique that optimizes segment distribution with minimal overhead, further improving performance across various operating points.
DTM-Codec achieves superior speech reconstruction quality by intelligently masking tokens and optimizing frame selection, challenging the limitations of traditional fixed-frame-rate codecs.
Variable frame rate (VFR) coding has recently emerged in neural speech codecs, allocating fewer frames to redundant regions and more frames to rapidly changing speech. VFR must transmit side information about retained time steps, but prior gains are either not rigorously addressed or often minor once these overhead bits are included in total bitrate. We present Dynamic Token Masking (DTM)-Codec, a neural speech codec that demonstrates clear gains over fixed-frame-rate baselines under a strict matched-total-bitrate protocol. DTM keeps selected encoder tokens, fills masked positions with a learned <MASK> embedding, and transmits a binary keep-mask for position-aware decoding. We further introduce Path Length Equalization (PLE), a linear-time boundary selector for VFR coding that yields well-spread adaptive segments with negligible overhead. Across operating points, DTM-Codec broadly improves reconstruction quality and intelligibility over fixed-frame-rate baselines.