Search papers, labs, and topics across Lattice.
This paper addresses the inefficiencies in data transfer between host and custom hardware accelerators by eliminating redundant memory copying in the AXI4MLIR framework. By implementing zero-copy data movement, the authors enhance the performance of linear algebra kernels, particularly in matrix-matrix multiplication tasks. The results demonstrate that this optimization can reduce main memory data movement by up to 2x, significantly improving accelerator utilization.
Eliminating redundant data copying can double memory efficiency and dramatically boost accelerator performance in machine learning workloads.
As custom hardware accelerators become increasingly central to machine learning workloads, efficient data transfer is critical for maximizing accelerator performance on linear algebra kernels. AXI4MLIR, an extension of the Multi-Level Intermediate Representation (MLIR) compiler framework for automated generation of host-accelerator driver code, incurs significant runtime overhead due to non-zero-copy CPU-accelerator data movement. During transfers from the host to the accelerator, data is copied from heap-allocated memory buffers into contiguous Direct Memory Access (DMA)-mapped buffers. This work identifies this copy as a redundant staging operation and eliminates it through zero-copy data movement. The optimization extends accel, an MLIR dialect introduced by AXI4MLIR, and implements lowering support that allocates buffers directly within DMA-mapped memory, thereby omitting the staging copy. We evaluate the proposed scheme using a configurable matrix-matrix multiplication accelerator and show that the zero-copy optimization reduces main memory data movement by up to 2x, increasing overall accelerator utilization.
