Search papers, labs, and topics across Lattice.
This paper addresses the limitations of the Ozaki scheme II's fast mode by proposing a revised scaling formula that maintains scale invariance and adheres to the uniqueness condition of the Chinese remainder theorem (CRT). The new method leverages the Cauchy鈥揝chwarz inequality to ensure that accuracy is preserved while achieving throughput levels similar to the original fast mode. Experimental results on an NVIDIA GH200 GPU demonstrate that the proposed approach achieves accuracy on par with the accurate mode without incurring additional computational overhead, thus enhancing the overall performance of the scaling process.
A new scale-invariant scaling formula for the Ozaki scheme II fast mode achieves high accuracy without sacrificing throughput, revolutionizing matrix multiplication efficiency.
Ozaki scheme II emulates high-precision matrix multiplication using low-precision integer matrix operations based on the Chinese remainder theorem (CRT). It first scales the high-precision matrices to convert them into integer matrices. For this scaling step, Ozaki scheme II provides two modes: accurate mode, which uses INT8 matrix multiplication to estimate scaling factors, and fast mode, which applies the Cauchy--Schwarz inequality at lower computational cost. We show that the existing formula lacks scale invariance; multiplying the input matrices by a constant changes the effective bit width of the integer matrices in the scaling step, causing accuracy degradation or CRT recovery failure. To address this, we propose a revised scaling formula derived from the CRT uniqueness condition via the Cauchy--Schwarz inequality. The proposed formula is scale-invariant by construction, guarantees that the CRT uniqueness condition is always satisfied, and introduces no additional overhead over the original fast mode. Experiments on an NVIDIA GH200 GPU show that the proposed method achieves accuracy comparable to that of accurate mode while maintaining throughput comparable to that of fast mode. In the accuracy--throughput trade-off, the proposed method overcomes the accuracy limitation of fast mode and the throughput constraint of accurate mode, offering a superior accuracy and performance.
