Search papers, labs, and topics across Lattice.
This paper introduces a novel structured pruning method for large language models (LLMs) that effectively integrates Adaptive Feature Retention (AFR) while overcoming challenges such as distribution mismatch, loss of sign information, and outlier influence. By employing power transformation for distribution alignment, sign-preserving score aggregation, and percentile-based outlier removal, the proposed approach achieves significant inference speedup without sacrificing accuracy. Experiments on Llama-3-8B, Vicuna-v1.5-13B, and LLaVA-v1.5-13B confirm that the method maintains performance levels comparable to unstructured pruning techniques.
Achieving structured pruning that rivals unstructured methods in accuracy while significantly accelerating inference speed could redefine efficiency benchmarks for large language models.
This paper proposes an improved structured pruning method for large language models (LLMs) that addresses key challenges in adapting Adaptive Feature Retention (AFR), an unstructured pruning technique, to structured pruning. When applying AFR to structured pruning, three major problems arise: distribution mismatch between heterogeneous pruning scores, loss of sign information indicating optimization direction consistency, and influence of outliers. To address these issues, we propose a unified approach combining power transformation for nonlinear distribution alignment, sign-preserving score aggregation, and percentile-based outlier removal. Experiments on Llama-3-8B, Vicuna-v1.5-13B, and LLaVA-v1.5-13B demonstrate that our method maintains accuracy comparable to unstructured pruning while achieving practical inference speedup through structured pruning.
