Search papers, labs, and topics across Lattice.
This study investigates the acceleration of a Pyannote 3.1-based speaker diarization pipeline on consumer hardware, focusing on the trade-off between speed and diarization error rate (DER). By adjusting the minimum cluster size relative to the embedding budget, the authors achieve significant speed improvements鈥攗p to 12.2x on AMI data鈥攚hile mitigating accuracy loss on in-the-wild datasets like VoxConverse. The proposed method recovers nearly 89% of the accuracy lost due to clustering issues, demonstrating a practical solution for on-device applications in speech technology.
Adapting the minimum cluster size to the embedding budget can recover nearly 89% of accuracy in speaker diarization while achieving over 12x speedup on consumer hardware.
Speech applications such as meeting transcription and voice agents would benefit from on-device speaker diarization, but practical adoption is limited by inference cost. We study how far a Pyannote 3.1-based pipeline can be accelerated on consumer hardware (an RTX 5070 Ti GPU and an Apple M4 laptop) while preserving diarization error rate (DER). A simple recipe: coarser segmentation stride and per-chunk embedding, yields multi-fold speedups and is DER-neutral on AMI, but degrades sharply on in-the-wild data: on VoxConverse, DER rises from 0.075 to 0.113. We trace the failure to speaker under-counting in the clustering stage, caused by a fixed minimum cluster size interacting with the reduced number of embeddings per speaker. We propose a relative minimum cluster size, mcs = round(f * n) with f = 0.01, which adapts to the embedding budget per recording. A single value of f recovers VoxConverse DER to 0.079 (about 89% of the lost accuracy) while keeping AMI flat, and the accelerated pipeline reaches up to 12.2x speedup on AMI (MPS) over our CAM++ baseline.
