MobileKernelBench: Can LLMs Write Efficient Kernels for Mobile Devices?

2026-03-12

AI Summary

This work addresses the high engineering complexity and data scarcity challenges in mobile operator generation by presenting the first systematic evaluation of large language models (LLMs) for CPU kernel code synthesis. The authors propose MoKA, a multi-agent generation framework that integrates repository-aware reasoning with a plan-and-execute mechanism, coupled with an automated compilation and performance validation pipeline. Evaluated on the newly introduced MobileKernelBench benchmark, MoKA achieves a 93.7% compilation success rate, with 27.4% of the generated kernels outperforming native implementations on the MNN CPU backend. These results demonstrate substantial improvements in both generation efficiency and runtime performance, highlighting the potential of LLM-driven approaches for practical mobile kernel development.

Technology Category

• Natural Language Processing: Code Generation / Program Synthesis from Natural Language
• Machine Learning: Large Multimodal Models (LMMs)
• Planning, Routing, and Scheduling: Planning with Language Models

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

Large language models (LLMs) have demonstrated remarkable capabilities in code generation, yet their potential for generating kernels specifically for mobile de- vices remains largely unexplored. In this work, we extend the scope of automated kernel generation to the mobile domain to investigate the central question: Can LLMs write efficient kernels for mobile devices? To enable systematic investigation, we introduce MobileKernelBench, a comprehensive evaluation framework comprising a benchmark prioritizing operator diversity and cross-framework interoperability, coupled with an automated pipeline that bridges the host-device gap for on-device verification. Leveraging this framework, we conduct extensive evaluation on the CPU backend of Mobile Neural Network (MNN), revealing that current LLMs struggle with the engineering complexity and data scarcity inher-ent to mobile frameworks; standard models and even fine-tuned variants exhibit high compilation failure rates (over 54%) and negligible performance gains due to hallucinations and a lack of domain-specific grounding. To overcome these limitations, we propose the Mobile K ernel A gent (MoKA), a multi-agent system equipped with repository-aware reasoning and a plan-and-execute paradigm.Validated on MobileKernelBench, MoKA achieves state-of-the-art performance, boosting compilation success to 93.7% and enabling 27.4% of generated kernelsto deliver measurable speedups over native libraries.

Problem

• LLM
• mobile kernel
• code generation
• performance optimization
• on-device execution

Innovation

• MobileKernelBench
• LLM code generation
• mobile kernel optimization
• multi-agent system
• on-device evaluation