Search papers, labs, and topics across Lattice.
This paper introduces an Integer Linear Programming (ILP)-based framework for optimizing the partitioning of machine learning workloads between CPUs and Computing-in-Memory (CIM) accelerators, addressing critical constraints of Resistive Random Access Memory (RRAM) such as limited memory and high write latency. By incorporating parallelism and leveraging both empirical profiling and analytical models, the framework minimizes end-to-end inference latency effectively. The results demonstrate significant performance improvements, achieving speedups of up to 30.9x over CPU-only execution on edge CPUs and 7.3x on high-performance CPUs, while providing valuable design insights for future CIM systems.
Achieving up to 30.9x speedup in ML inference by optimizing workload partitioning between CPUs and CIM accelerators reveals the untapped potential of heterogeneous computing.
Computing-in-Memory (CIM) accelerators execute Matrix-Vector Multiplications (MVMs) in memory, making them a compelling solution for Machine Learning (ML) workloads. However, existing ML workload partitioning approaches for CIM accelerators do not fully account for Resistive Random Access Memory (RRAM) constraints such as limited memory, high write latency, and limited endurance. They also neglect parallelism, low-level architectural effects, or the Central Processing Unit (CPU) as a complementary compute resource. To address these limitations, we propose an Integer Linear Programming (ILP)-based workload partitioning framework for heterogeneous CPU-CIM systems. It minimizes end-to-end inference latency under RRAM constraints, captures parallelism, and combines empirical profiling with analytical models. Using our framework, heterogeneous CPU-CIM execution achieves speedups of up to 30.9x over CPU-only execution on an edge CPU and 7.3x over a high-performance CPU. A Design Space Exploration (DSE) yields further design insights for future CIM accelerators.
