Search papers, labs, and topics across Lattice.
This paper explores the application of model pruning to enhance the computational efficiency of AudioLDM, a U-Net-based text-to-audio generative model. By analyzing parameter redundancy and employing a filter-pruning strategy guided by norm-based criteria, the authors achieve a reduction of up to 83% in parameters and 39% in multiply-accumulate operations while maintaining or even improving audio generation quality. Notably, the study reveals that while pruning impacts the generation of certain critical sound events, these can often be recovered through lightweight finetuning.
Pruning can slash the computational cost of text-to-audio models by over 80% without sacrificing quality, but it poses risks to generating critical sound events.
Diffusion-based text-to-audio generative models such as AudioLDM achieve high perceptual quality and strong semantic consistency; however, their practical deployment is hindered by the substantial computational cost of the U-Net denoising backbone. In this work, we apply model pruning to improve the computational efficiency of AudioLDM, a U-Net-based text-conditioned audio latent diffusion model. We analyse parameter redundancy across U-Net convolutional blocks and evaluate a filter-pruning strategy. Pruning is guided by norm-based criteria and followed by lightweight finetuning to recover performance losses. Experimental results demonstrate that up to 83% of the parameters and 39% of the multiply-accumulate operations of U-Net have been reduced while maintaining, and in some cases improving, generation quality compared to the baseline unpruned network. We find that pruning affects AudioLDM's ability to generate certain sound events including safety-critical sounds such as gunshots, sirens, and explosions, as well as mechanical sounds such as drills and sewing machines, and other sounds such as sprays and tick-tocks, which are mostly recovered by lightweight finetuning of the pruned model.
