Search papers, labs, and topics across Lattice.
The paper introduces Hybrid Quantization and Pruning (HQP), a framework for edge AI model compression that integrates sensitivity-aware structural pruning with post-training quantization. HQP uses a dynamic weight sensitivity metric, approximated from the Fisher Information Matrix, to guide filter pruning subject to a maximum accuracy drop constraint before 8-bit quantization. Experiments on MobileNetV3 and ResNet-18 on NVIDIA Jetson platforms show HQP achieves up to 3.12x speedup and 55% model size reduction with less than 1.5% accuracy drop, outperforming single-objective compression methods.
Get 3x faster and half the size on edge devices without sacrificing accuracy by intelligently combining pruning and quantization.
The escalating demand for high-fidelity, real-time inference in distributed edge-cloud environments necessitates aggressive model optimization to counteract severe latency and energy constraints. This paper introduces the Hybrid Quantization and Pruning (HQP) framework, a novel, integrated methodology designed to achieve synergistic model acceleration while adhering to strict quality guarantees. We detail a sensitivity-aware structural pruning algorithm that employs a dynamic weight sensitivity metric, derived from a highly efficient approximation of the Fisher Information Matrix (FIM), to guide the iterative removal of redundant filters. This pruning is strictly conditional, enforcing an adherence to a maximum permissible accuracy drop (Delta ax) before the model proceeds to 8-bit post-training quantization. This rigorous coordination is critical, as it ensures the resultant sparse model structure is maximally robust to quantization error and hardware-specific kernel optimization. Exhaustive evaluation across heterogeneous NVIDIA Jetson edge platforms, utilizing resource-efficient architectures like MobileNetV3 and ResNet-18, demonstrates that the HQP framework achieves a peak performance gain of 3.12 times inference speedup and a 55 percent model size reduction, while rigorously containing the accuracy drop below the 1.5 percent constraint. A comprehensive comparative analysis against conventional single-objective compression techniques validates the HQP framework as a superior, hardware-agnostic solution for deploying ultra-low-latency AI in resource-limited edge infrastructures.
