Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces DMA-KWS, a user-defined keyword spotting framework that addresses challenges in discriminability, speaker variance, and data costs. It employs a dual-stage matching pipeline (CTC decoding + QbyT with phoneme matcher) for improved accuracy with confusable words, multi-modal enrollment fusing speech and text embeddings for user-specific adaptation, and parameter-efficient continual adaptation using synthetic and real data. Experiments on LibriPhrase Hard demonstrate state-of-the-art performance (97.85% AUC, 6.13% EER) and significant gains in speaker-dependent settings with only 187k updated parameters.
User-defined keyword spotting can now achieve state-of-the-art accuracy and adapt to individual speakers with minimal data and parameter updates, thanks to a novel dual-stage matching and multi-modal enrollment framework.
User-defined keyword spotting (KWS) is crucial for personalized voice interaction, yet existing methods face several challenges: (1) insufficient discriminability among confusable words, (2) performance inconsistency across speakers with varying pronunciations, and (3) high data cost to ensure reliable wake-word performance. In this paper, we introduce DMA-KWS, an efficient and robust framework for user-defined keyword spotting. First, it adopts a dual-stage matching pipeline: CTC decoding with streaming phoneme search to locate candidate segments, followed by QbyT with a phoneme matcher for fine-grained verification, enabling it to better distinguish confusable words. Next, multi-modal enrollment fuses user-specific speech with text embeddings to further improve accuracy for registered users. Finally, a parameter-efficient continual adaptation mechanism performs lightweight updates using synthetic and real data. Extensive experiments demonstrate the superior performance of DMA-KWS. On the LibriPhrase Hard subset, it achieves 97.85% AUC and 6.13% EER, reaching state-of-the-art performance. In speaker-dependent settings, DMA-KWS consistently outperforms text-only enrollment, demonstrating significant performance gains. Moreover, the proposed parameter-efficient fine-tuning mechanism adapts DMA-KWS with only 187k updated parameters, further enhancing KWS performance while ensuring suitability for on-device deployment.