Search papers, labs, and topics across Lattice.
This paper introduces MobileKernelBench, a framework for evaluating LLMs' ability to generate efficient kernels for mobile devices, focusing on the MNN CPU backend. Experiments reveal that standard and fine-tuned LLMs struggle with the complexities of mobile frameworks, exhibiting high compilation failure rates and limited performance gains. To address this, the authors propose Mobile Kernel Agent (MoKA), a multi-agent system with repository-aware reasoning and a plan-and-execute paradigm, which significantly improves compilation success and kernel performance.
LLMs can now generate mobile kernels that outperform native libraries 27.4% of the time, thanks to a new multi-agent system that tackles compilation failures and hallucinations.
Large language models (LLMs) have demonstrated remarkable capabilities in code generation, yet their potential for generating kernels specifically for mobile de- vices remains largely unexplored. In this work, we extend the scope of automated kernel generation to the mobile domain to investigate the central question: Can LLMs write efficient kernels for mobile devices? To enable systematic investigation, we introduce MobileKernelBench, a comprehensive evaluation framework comprising a benchmark prioritizing operator diversity and cross-framework interoperability, coupled with an automated pipeline that bridges the host-device gap for on-device verification. Leveraging this framework, we conduct extensive evaluation on the CPU backend of Mobile Neural Network (MNN), revealing that current LLMs struggle with the engineering complexity and data scarcity inher-ent to mobile frameworks; standard models and even fine-tuned variants exhibit high compilation failure rates (over 54%) and negligible performance gains due to hallucinations and a lack of domain-specific grounding. To overcome these limitations, we propose the Mobile K ernel A gent (MoKA), a multi-agent system equipped with repository-aware reasoning and a plan-and-execute paradigm.Validated on MobileKernelBench, MoKA achieves state-of-the-art performance, boosting compilation success to 93.7% and enabling 27.4% of generated kernelsto deliver measurable speedups over native libraries.
