Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces the Tensor Memory Engine (TME), a hardware/software co-designed system that reorganizes data on-the-fly to improve cache locality for data-intensive applications. By decoupling computation from memory access and inserting the TME into the CPU's data path, the system provides applications with data layouts optimized for spatiotemporal locality. The TME, implemented on SoC/FPGA platforms, aims to mitigate the memory wall challenge without requiring costly application redesigns or inflated memory footprints.
Get near-ideal cache locality without redesigning your application or blowing up your memory footprint: a hardware/software engine reorganizes data on the fly.
The shift to data-intensive processing from the cloud to the edge has introduced new challenges and expectations for the next generation of intelligent computing systems. As the memory wall continues to grow, modern systems can only meet these performance expectations by displaying data access patterns that exhibit ideal layouts in memory and ideal spatiotemporal locality in caches. However, only a few data-intensive applications are characterized by ideal locality. Instead, most applications exhibit either (i) poor locality when naively implemented and must undergo costly redesigns and tuning or (ii) inflated memory footprint to offer proper locality. To address the aforementioned challenges, we propose a hardware/software co-designed approach that can be implemented on commercially available SoC/FPGA platforms. Our approach seamlessly inserts in the CPUs'data path a Tensor Memory Engine that provides data with an ideal cache locality to running applications by (i) accessing the memory on behalf of the CPUs and (ii) composing a re-organized view of the memory layout. Unlike in- and near-memory computing approaches, it sets itself apart by clearly decoupling computing and memory accesses; computation is still performed on CPUs while the data re-organization is delegated to the Tensor Memory Engine.