Search papers, labs, and topics across Lattice.
This paper introduces Normalization Affine Preconditioning (NAP), a novel approach for improving low-bit quantization in neural networks by optimizing low-dimensional high-leverage subspaces, specifically targeting normalization affine parameters. By freezing backbone weights and fine-tuning only affine parameters during post-training quantization (PTQ) and employing an alternating schema for quantization-aware training (QAT), NAP effectively enhances quantization robustness and mitigates accuracy degradation. Experimental results on ImageNet and CIFAR-100 demonstrate that NAP significantly improves performance in low-bit quantization scenarios, surpassing traditional methods with minimal tuning costs.
Targeted optimization of normalization affine parameters can dramatically enhance low-bit quantization performance, breaking the limits of conventional training methods.
Low-bit quantization suffers severe accuracy degradation on compact networks, rooted in the dominant full-parameter coupled training paradigm that ignores parameter subspace heterogeneity. Their limited feature redundancy leaves little room to absorb quantization errors. Conventional pipelines adopt monolithic optimization: PTQ reconstructs fixed pretrained models without improving inherent quantization friendliness; QAT updates all parameters jointly, suffering from gradient coupling between backbone weights and calibration parameters. In this paper, we identify normalization affine parameters as a low-dimensional high-leverage subspace dominating quantization robustness, and propose Normalization Affine Preconditioning (NAP) for targeted subspace optimization. For PTQ, NAP freezes backbone weights and fine-tunes only affine parameters under the target fake-quantization graph on full-precision models, proactively boosting quantization friendliness before downstream reconstruction. For QAT, we introduce an alternating QAT-NAP schema that decouples feature learning and numerical calibration, breaking the performance ceiling of saturated joint training. Theoretical analysis confirms BN affine parameters fully cancel the channel-wise affine component of quantization distortion, while nonlinear rounding and clipping residuals form the irreducible error boundary; distillation-guided NAP acts as directional flatness optimization, projecting teacher-student logit mismatch onto the restricted subspace. Experiments on ImageNet and CIFAR-100 show NAP recovers severely collapsed low-bit quantization, consistently boosts reconstruction-based PTQ, and outperforms saturated full-parameter QAT with negligible tuning cost. This work reveals the principle of targeted low-dimensional subspace optimization, offering a new perspective beyond full-parameter coupled training for efficient deep learning.
