Search papers, labs, and topics across Lattice.
This paper introduces a data-free and training-free compression method for speech foundation models that employs channelwise clustering through k-means. By implementing mixed sparsity pruning with a layer-level varying number of parameter clusters, the authors achieved significant word error rate (WER) reductions on the LibriSpeech dataset, outperforming traditional magnitude-based pruning techniques. Specifically, HuBERT-large demonstrated WER reductions of 27.73%/18.61% absolute before fine-tuning and 0.19%/0.79% absolute after fine-tuning, while Whisper-large-v3 achieved similar gains at 10% sparsity, all without increasing WER relative to the uncompressed models.
Achieving up to 27.73% absolute WER reduction without any training or additional data challenges the conventional reliance on fine-tuning for model compression.
This paper presents a novel data-free and training-free compression approach for speech foundation models using channelwise clustering via k-means. More fine-grained, mixed sparsity pruning by layer-level varying number of parameter clusters is also explored. Experiments conducted on the LibriSpeech dataset suggest that when operating with pruning sparsity of 50% on HuBERT-large, consistent WER reductions of 27.73%/18.61% absolute (34.37%/21.91% relative) over the magnitude-based pruning were obtained on the test-clean and test-other subsets before fine-tuning and 0.19%/0.79% absolute (3.36%/4.62% relative) after fine-tuning with only 3 epochs. Similar WER reductions of 2.86%/5.02% absolute (59.21%/55.29% relative) were observed against magnitudebased pruning on Whisper-large-v3 at 10% sparsity, all with no significant WER increase relative to the uncompressed baseline.
