Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
LP-GEMM decomposes the GEMM kernel to enable packing-layout propagation across sequential GEMM operations, eliminating redundant data repacking common in BLAS API usage. This approach maintains BLAS semantic correctness while optimizing for sequences of dependent GEMMs, typical in ML workloads. Benchmarks on x86 (AVX-512) and RISC-V (RVV 1.0) show an average 2.25x speedup over OpenBLAS on Intel x86 and competitive performance against Intel MKL, validated by a Llama-3.2 inference implementation.
Stop wasting cycles on redundant matrix packing: LP-GEMM unlocks 2x+ speedups for sequential GEMMs by propagating layouts across operations.
In Scientific Computing and modern Machine Learning (ML) workloads, sequences of dependent General Matrix Multiplications (GEMMs) often dominate execution time. While state-of-the-art BLAS libraries aggressively optimize individual GEMM calls, they remain constrained by the BLAS API, which requires each call to independently pack input matrices and restore outputs to a canonical memory layout. In sequential GEMMs, these constraints cause redundant packing and unpacking, wasting valuable computational resources. This paper introduces LP-GEMM, a decomposition of the GEMM kernel that enables packing-layout propagation across sequential GEMM operations. This approach eliminates unnecessary data repacking while preserving full BLAS semantic correctness at the boundaries. We evaluate LP-GEMM on x86 (AVX-512) and RISC-V (RVV 1.0) architectures across MLP-like and Attention-like workloads. Our results show average speedups of 2.25x over OpenBLAS on Intel x86 for sequential GEMMs and competitive gains relative to vendor-optimized libraries such as Intel MKL. We demonstrate the practicality of the approach beyond microbenchmarks by implementing a standalone C++ version of the Llama-3.2 inference path using exclusively BLAS-level GEMM calls. These results confirm that leveraging data layout propagation between operations can significantly boost performance.