Search papers, labs, and topics across Lattice.
PrunePath is introduced as a budget-adaptive structured sparsification framework for feed-forward network (FFN) layers in language models, addressing the challenge of converting sparsity into hardware efficiency. It replaces independent expert-wise thresholding with a softmax-normalized routing distribution, activating experts based on a cumulative-mass threshold, thereby imposing a token-level probability budget. Experiments across NLU, NLG, and instruction-tuning show PrunePath achieves a better sparsity-performance trade-off, with Triton kernels demonstrating memory savings and decoding speed improvements.
Unlock practical memory savings and faster decoding in LLMs with PrunePath, a structured sparsification method that adaptively activates experts based on token-level probability budgets.
Feed-forward networks (FFNs) dominate the parameter count and computation of modern language models, yet existing pruning methods often struggle to convert sparsity into hardware-friendly inference efficiency gains. We introduce \textbf{PrunePath}, a budget-adaptive structured sparsification framework for FFN layers. Built on MoEfication, PrunePath replaces independent expert-wise thresholding with a softmax-normalized routing distribution and activates important experts under a cumulative-mass threshold. This formulation imposes a token-level probability budget, enabling adaptive expert counts and a direct inference-time sparsity knob from a single checkpoint. Across NLU, NLG, and instruction-tuning evaluations, PrunePath achieves a favorable sparsity--performance trade-off compared with existing static pruning and MoEfication-based methods. We further implement Triton kernels for KV-cache decoding to translate the resulting structured sparsity into practical memory savings and measurable decoding-speed improvements. These results demonstrate the superior performance of PrunePath for building highly sparse, deployment-friendly large language models.
