Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces wav2tok 2.0, an advanced audio tokenization method designed to enhance query-by-example spoken term detection (QbE-STD) by maintaining explicit pairwise token alignment. The approach employs a two-stage training process that first utilizes contrastive learning for speaker-invariant representation learning, followed by enforcing token consistency through a combination of CTC alignment loss and a novel DTW-aligned framewise prediction objective. Experimental results demonstrate that wav2tok 2.0 significantly outperforms existing methods like BEST-STD and general-purpose tokenizers, achieving both efficiency and scalability in audio retrieval tasks.
wav2tok 2.0 achieves superior performance in spoken term detection while maintaining scalability, thanks to its innovative two-stage training process.
Learning discrete speech representations that preserve similarity across variable-length utterances is central to query-by-example spoken term detection (QbE-STD). While wav2tok introduced CTC-based sequence alignment to enforce token consistency, its tightly coupled clustering and alignment training recipe limits scalability. We propose wav2tok 2.0, a scalable alignment-aware speech tokenizer built on the BEST-STD backbone. wav2tok 2.0 employs staged training, first learning discriminative, speaker-invariant representations via contrastive learning and vector quantization, and then enforcing pairwise token consistency using a CTC alignment loss and a novel DTW-aligned framewise prediction objective with adaptive weighting. Experiments show that wav2tok 2.0 consistently outperforms BEST-STD and general-purpose tokenizers on QbE-STD while remaining efficient and scalable.