Track genomics, neuroscience and molecular biology.
Search papers, labs, and topics across Lattice.
The paper introduces multi-format quantization-aware training (MF-QAT) to train a single model robust to multiple quantization formats, addressing the need for flexible precision in diverse deployment scenarios. They demonstrate that MF-QAT achieves performance comparable to single-format QAT for each target precision, even for unseen formats. They further propose a "Slice-and-Scale" conversion method enabling on-the-fly conversion to lower MXINT/MXFP formats from a single anchor checkpoint (MXINT8/MXFP8) with minimal accuracy loss.
Train one quantization-aware model to rule them all: MF-QAT matches single-format QAT performance across precisions, and "Slice-and-Scale" enables runtime precision switching from a single checkpoint.
Quantization-aware training (QAT) is typically performed for a single target numeric format, while practical deployments often need to choose numerical precision at inference time based on hardware support or runtime constraints. We study multi-format QAT, where a single model is trained to be robust across multiple quantization formats. We find that multi-format QAT can match single-format QAT at each target precision, yielding one model that performs well overall across different formats, even formats that were not seen during training. To enable practical deployment, we propose the Slice-and-Scale conversion procedure for both MXINT and MXFP that converts a high-precision representation into lower-precision formats without re-training. Building on this, we introduce a pipeline that (i) trains a model with multi-format QAT, (ii) stores a single anchor format checkpoint (MXINT8/MXFP8), and (iii) allows on-the-fly conversion to lower MXINT or MXFP formats at runtime with negligible-or no-additional accuracy degradation. Together, these components provide a practical path to elastic precision scaling and allow selecting the runtime format at inference time across diverse deployment targets.