Search papers, labs, and topics across Lattice.
This paper introduces COSM, a cooperative scheduling framework that enables concurrent execution of Processing-in-Memory (PIM) and CPU tasks on mobile devices, addressing the challenges posed by bank conflicts and bus congestion. By implementing a low-interference PIM control interface and an idleness-aware scheduling method, COSM maximizes PIM command generation while minimizing disruption to CPU memory accesses. Experimental results show that COSM enhances PIM throughput by up to 2.8 times compared to baseline methods, with negligible impact on CPU performance, achieving less than 2.0% loss.
COSM achieves a remarkable 2.8x improvement in PIM throughput while keeping CPU performance degradation under 2.0%.
The development of on-device large language models (LLMs) is driven by the need for privacy and fast response times. Energy-intensive data transfer on mobile devices makes Processing-in-Memory (PIM) an effective solution. Due to stringent DRAM cost constraints, limited physical footprint on circuit boards, and the interaction between applications and LLMs, it is imperative for the CPU and PIM to operate concurrently within a shared memory space. However, challenges such as bank conflicts and bus congestion can arise, potentially diminishing the performance and energy benefits of PIM. To address this challenge, we introduce COSM, a cooperative scheduling framework designed to facilitate the concurrent operation of PIM and CPU tasks on mobile platforms. Our key innovations include: 1) a low-interference PIM control interface that generates the maximum number of PIM commands without disrupting CPU memory accesses; 2) an idleness-aware scheduling method that integrates PIM commands into available idle time windows within the CPU's access sequence. COSM not only hides PIM execution latency from the CPU, but also overlaps PIM execution with data transfer. Experiments on concurrent execution of LLMs and mobile workloads, including mobile applications and compute-intensive kernels, demonstrate that COSM improves PIM throughput by up to 2.8x compared to the baseline scheduling method with less than 2.0% CPU performance loss.
Tsinghua AI